{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "92010cd5",
   "metadata": {},
   "source": [
    "# 2.3 优化提示词改善答疑机器人回答质量"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0b18130",
   "metadata": {},
   "source": [
    "## 🚄 前言"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a33d28f-9014-414a-a7b5-b016e12d4bb0",
   "metadata": {},
   "source": [
    "上一节，你学习了**上下文工程 (Context Engineering)** 的一种关键技术——RAG，通过为大模型提供外部知识来扩展其能力。但这仅仅是构建优秀用户体验的第一步。用户往往期待更个性化、更精准的交互。\n",
    "\n",
    "本节将深入探索上下文工程的另一重要方面：如何通过优化**提示词 (Prompt)** 来精细地控制大模型的输出，例如调整语气、规范格式，甚至赋予它处理文本总结、推断、转换等多种任务的能力。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37a39a58",
   "metadata": {},
   "source": [
    "## 🍁 课程目标\n",
    "\n",
    "学完本节课程后，你将能够：\n",
    "\n",
    "* 了解提示词框架和模板\n",
    "* 了解提示词技巧和最佳实践\n",
    "* 学习在工程中应用大模型处理多种不同类型的任务"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b0e80fa",
   "metadata": {},
   "source": [
    "## 1. 前文回顾\n",
    "\n",
    "上一节中通过RAG方法，大模型已经获取到了公司的私有知识。为了方便调用，将其封装成了几个函数，并保存在了 chatbot/rag.py 中。 现在你可以通过如下代码来快速调用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7aa6756d-47a4-4b11-8074-d992d6ce327b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:49:03.577636Z",
     "start_time": "2025-01-15T06:49:03.573248Z"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# 环境变量中指定 base_url\n",
    "os.environ[\"BASE_URL\"] = \"https://dashscope.aliyuncs.com/compatible-mode/v1\"\n",
    "\n",
    "from config.load_key import load_key\n",
    "# 加载API密钥\n",
    "load_key()\n",
    "print(f'''你配置的 API Key 是：{os.environ[\"DASHSCOPE_API_KEY\"][:5]+\"*\"*5}''')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91138451-fbc1-49ff-a268-2ece341b10bd",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:49:11.770743Z",
     "start_time": "2025-01-15T06:49:11.611835Z"
    }
   },
   "outputs": [],
   "source": [
    "from chatbot import rag, llm\n",
    "# 加载索引\n",
    "# 上小节已经建立了索引，因此这里可以直接加载索引。如果需要重建索引，可以增加一行代码：rag.indexing()\n",
    "index = rag.load_index()\n",
    "query_engine = rag.create_query_engine(index=index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d2d8d11-5a07-441c-9cd6-6ba10bf80603",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:49:16.961802Z",
     "start_time": "2025-01-15T06:49:16.959373Z"
    }
   },
   "outputs": [],
   "source": [
    "# 定义问答函数\n",
    "def ask_llm(question, query_engine):\n",
    "    streaming_response = query_engine.query(question)\n",
    "    streaming_response.print_response_stream()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1715f36",
   "metadata": {},
   "source": [
    "## 2. 优化提示词以改善大模型回答质量\n",
    "\n",
    "上一节，你已经成功地让答疑机器人通过 RAG 掌握了公司的内部知识。现在，你可以测试一下它的实际效果。一位新同事想了解项目管理工具，于是他向机器人发起了提问：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c51ba4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 仅使用RAG时，机器人的表现\n",
    "question = \"我们公司项目管理应该用什么工具？\"\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16c5e4d9",
   "metadata": {},
   "source": [
    "预期输出（示例）：\n",
    "```\n",
    "公司项目管理可以使用Jira或Trello等项目管理软件。这些工具能够帮助团队更好地规划和跟踪项目的进度，确保任务按时完成。同时，它们还支持任务分配、时间跟踪和团队协作等功能，有助于提高工作效率。\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0d5c9f2",
   "metadata": {},
   "source": [
    "这个回答是正确的，但不够“好用”。同事反馈说：“如果能直接附上工具的链接就完美了，省得我再去到处找。”\n",
    "\n",
    "这个需求非常合理。一个最直观、最“暴力”的解决方法，或许是在用户的问题后面，手动加上一句指令："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b12a310a-660c-4cd3-9d23-8819bdefb8bb",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:49:34.992776Z",
     "start_time": "2025-01-15T06:49:29.417135Z"
    }
   },
   "outputs": [],
   "source": [
    "# 尝试一个简单的“补丁”\n",
    "question = \"我们公司项目管理应该用什么工具？\"\n",
    "instruction = \" 回答时请务必附带上工具的官方网站或下载链接。\"\n",
    " \n",
    "# 将问题和指令简单地拼接在一起\n",
    "new_question = question + instruction\n",
    "ask_llm(new_question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59c07db0",
   "metadata": {},
   "source": [
    "预期输出（示例）：\n",
    "```\n",
    "对于项目管理，推荐使用 Jira 或 Trello。这两个工具都非常适合管理和跟踪项目进度，确保团队成员之间的高效协作。\n",
    "\n",
    "- **Jira** 是一个非常强大的项目管理工具，特别适用于软件开发项目。它可以帮助你跟踪问题和任务，管理项目进度。你可以访问 [Atlassian官网](https://www.atlassian.com/software/jira) 获取更多信息或下载。\n",
    "\n",
    "- **Trello** 则是一个更加灵活的看板式项目管理工具，适合各种类型的项目。通过卡片和列表的形式，可以直观地看到项目的进展。你可以访问 [Trello官网](https://trello.com/) 获取更多信息或注册使用。\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "806fadd6-673a-4542-b3e2-fe36310b30ad",
   "metadata": {},
   "source": [
    "问题似乎解决了。但……真的解决了吗？\n",
    "\n",
    "你需要考虑另一个场景，当用户问一个跟工具无关的问题时："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0260a2af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 看看这个“补丁”在其他问题上的表现\n",
    "question_2 = \"我们公司的年假有多少天？\"\n",
    "instruction = \" 回答时请务必附带上工具的官方网站或下载链接。\"\n",
    " \n",
    "new_question_2 = question_2 + instruction\n",
    "ask_llm(new_question_2, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7855091",
   "metadata": {},
   "source": [
    "预期输出（示例）：\n",
    "```\n",
    "您的问题涉及到具体的公司福利政策，特别是年假天数，但提供的信息中并没有直接提到这一点。通常这类详细信息会记录在员工手册或者公司内部的人力资源政策文件中。建议您直接联系人力资源部获取准确的信息。人力资源部的联系人是熊伟，您可以发送邮件至xiongwei@educompany.com询问具体事宜。\n",
    "\n",
    "至于您要求的工具官方网站或下载链接，由于您的问题并未提及需要使用哪个工具，因此无法提供相关链接。\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c76e76cc",
   "metadata": {},
   "source": [
    "现在，问题暴露了。这种“一刀切”地拼接指令的方式非常脆弱。它不仅可能让模型的回答变得啰嗦和奇怪，而且每次提问都要手动判断该拼接哪句指令，这在真实的应用中是完全不可行的。\n",
    "\n",
    "**这正是深入学习提示词工程（Prompt Engineering）的原因**。 它属于 **上下文工程（Context Engineering）** 的一部分，教你如何不再使用这种临时的“补丁”，而是系统性地、智能地构建与模型的对话上下文，精确地“指导”模型如何根据不同的情况来回答问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c006a5ea-5693-4805-9d2c-ccc0d2214939",
   "metadata": {},
   "source": [
    "## 3. 提示词框架\n",
    "\n",
    "### 3.1 基本要素\n",
    "\n",
    "当和大模型在交流时，可以将它想象是一个经过“社会化训练的”人，交流方式应当和人与人之间传递信息的方式一样。你的需求需要清晰明确，不能有歧义。你的提问方式（Prompt）越清晰明确，大模型越能抓住问题的关键点，回复就越符合你的预期。为了系统性地构建高效的上下文，我们可以遵循一个包含多个基本要素的提示词框架：**任务目标、上下文、角色、受众、样例、输出格式**。这些要素共同构成了一个完整的上下文“蓝图”，能帮助你构建一个完整、有效的提示词。\n",
    "\n",
    "|要素|含义|\n",
    "|----|----|\n",
    "|任务目标（Object）|明确要求大模型完成什么任务，让大模型专注具体目标|\n",
    "|上下文（Context）|任务的背景信息，比如操作流程、任务场景等，明确大模型理解讨论的范围|\n",
    "|角色（Role）|大模型扮演的角色，或者强调大模型应该使用的语气、写作风格等，明确大模型回应的预期情感|\n",
    "|受众（Audience）|明确大模型针对的特定受众，约束大模型的应答风格|\n",
    "|样例（Sample）|让大模型参考的具体案例，大模型会从中抽象出实现方案、需要注意的具体格式等信息|\n",
    "|输出格式（Output Format）|明确指定输出的格式、输出类型、枚举值的范围。通常也会明确指出不需要输出的内容和不期望的信息，可以结合样例来进一步明确输出的格式和输出方法|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc35341a-3d7c-4881-81e3-935d42a1dcb2",
   "metadata": {},
   "source": [
    "当然，除了上面讲的提示词框架，许多问题分析的思维范式都可以用来帮助你描述清晰具体的需求。例如，SWOT分析法、5W2H分析法等。另外，你也可以考虑使用阿里云百炼提供的[提示词自动优化工具](https://bailian.console.aliyun.com/?tab=app#/component-manage/prompt/optimize)，来帮助你完善提示词。\n",
    "\n",
    "<a href=\"https://img.alicdn.com/imgextra/i3/O1CN014JCCqn22zX6xB4tt3_!!6000000007191-0-tps-2068-1052.jpg\" target=\"_blank\">\n",
    "<img src=\"https://img.alicdn.com/imgextra/i3/O1CN014JCCqn22zX6xB4tt3_!!6000000007191-0-tps-2068-1052.jpg\" width=\"700\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e1fa6d9",
   "metadata": {},
   "source": [
    "### 3.2 提示词模板\n",
    "\n",
    "在开发大模型应用时，直接让用户根据框架书写提示词并非最佳选择。你可以参考各种提示词框架中的要素，构建一个提示词模板。提示词模板可以预设部分信息，如大模型的角色、注意事项等，以此来约束大模型的行为。开发者只需在模板中配置输入参数，便能创建标准化的大模型的应用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a84e29a3-a017-47c4-907e-2ff897333d1d",
   "metadata": {},
   "source": [
    "使用 LlamaIndex 中创建的 RAG应用中，有个默认的提示词模板，如下所示：\n",
    "\n",
    "- 默认的模板可以使用代码查看，你可以参考[LlamaIndex官网的代码。](https://developers.llamaindex.ai/python/examples/prompts/prompt_mixin/#customize-the-prompt)LlamaIndex原始prompt模板为：\n",
    "\n",
    "```text\n",
    "Context information is below.\n",
    "---------------------\n",
    "{context_str}\n",
    "---------------------\n",
    "Given the context information and not prior knowledge, answer the query.\n",
    "Query: {query_str}\n",
    "Answer:\n",
    "```\n",
    "\n",
    "其中，`context_str`和`query_str`都表示变量。在进行向量检索和提问过程中，`context_str`将替换为从向量库中检索到的上下文信息，`query_str`则替换为用户的问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78ff8788-4d67-47b8-a4a2-3f3a85e6b0c5",
   "metadata": {},
   "source": [
    "由于原模板是通用模板，不适合用来约束答疑机器人的行为。你可以通过下列示例代码重新调整提示词模板，其中`prompt_template_string`表示新的提示词模板，你可以根据自己的场景自行修改。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05aea996-fdc8-4a35-922f-288313881fa3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T07:52:22.442975Z",
     "start_time": "2025-01-15T07:52:22.435217Z"
    }
   },
   "outputs": [],
   "source": [
    "# 构建提示词模板\n",
    "prompt_template_string = (\n",
    "    \"你是公司的客服小蜜，你需要简明扼要的回答用户的问题\"\n",
    "    \"【注意事项】：\\n\"\n",
    "    \"1. 依据上下文信息来回答用户问题。\\n\"\n",
    "    \"2. 你只需要回答用户的问题，不要输出其他信息\\n\"\n",
    "    \"以下是参考信息。\"\n",
    "    \"---------------------\\n\"\n",
    "    \"{context_str}\\n\"\n",
    "    \"---------------------\\n\"\n",
    "    \"问题：{query_str}\\n。\"\n",
    "    \"回答：\"\n",
    ")\n",
    "\n",
    "# 更新提示词模板\n",
    "rag.update_prompt_template(query_engine,prompt_template_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa99f324",
   "metadata": {},
   "source": [
    "## 4. 构建有效提示词的技巧\n",
    "\n",
    "在3.1中列举了一些提示词设计中的要素，接下来将从提示词要素出发，结合具体场景展开讲解提示词技巧。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1b3eb55-646d-41dd-b801-152d76205bbe",
   "metadata": {},
   "source": [
    "### 4.1 清晰表达需求，并使用分隔符\n",
    "\n",
    "明确的表达需求可以确保大模型生成的内容与任务高度相关。需求包括任务目标、背景及上下文信息，还可以使用分隔符将各种提示词要素隔开。\n",
    "\n",
    "分隔符可以使大模型抓住具体的目标，避免模糊的理解，也减少对不必要信息的处理。分隔符一般可以选择 “**【】**”、“**<< >>**”、“**###**”来标识关键要素，用“**===**”、“**---**”来分隔段落，或者使用xml标签如`<tag> </tag>`来对特定段落进行标识。当然，分隔符不止上述提到的几种，只需要起到明确阻隔的作用即可。需要注意的是，如果提示词中已大量使用某种符号（如【】），则应避免用该符号作为分隔符，以防混淆。\n",
    "\n",
    "在下面的示例中，将基于3.2中的提示词模板，以答疑机器人的身份帮你完成文档润色。\n",
    "\n",
    "- 你需要运行3.2中的代码，才能使新的提示词模板生效。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8918731d-49b6-4132-9ba5-2dce3eee360c",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T07:52:32.192934Z",
     "start_time": "2025-01-15T07:52:24.619420Z"
    }
   },
   "outputs": [],
   "source": [
    "question = \"\"\"\n",
    "把下列【】括起来的文本进行扩写和润色，让文案生动且富有创造力，并且受到公司新员工的喜欢。\n",
    "【新员工训练营活动】\n",
    "\"\"\"\n",
    "ask_llm(question,query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62ae4c28-a2e6-4ad3-995b-7af2d78b19d1",
   "metadata": {},
   "source": [
    "由上述运行结果可以发现，提示词中明确了任务需求：对文本进行扩写和润色，任务主题为“**新员工训练营活动**”，受众为“**公司员工**”，并使用了“**【】**”分隔符分隔了文本。使用这样的提示词，让输出的文本不仅保持了原本的含义，还更加生动有趣。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7e4407d-3cff-4f00-a39b-b8ddfd7cce1d",
   "metadata": {},
   "source": [
    "### 4.2 限定角色和受众\n",
    "\n",
    "角色指大模型在特定场景下应扮演的身份，如专家、顾问或助手。受众是使用模型输出的目标用户群体，如普通消费者或学生。这有助于开发者定制内容的风格和深度。以下示例将展示不同的系统角色对输出文本风格和内容的影响，用户的问题为：qwen-vl是什么？\n",
    "\n",
    "- 以下仅演示提示词模板和输出的内容，你可以将提示词模板传入到`prompt_template_string`中，将用户问题传入到`question`中，再调用`ask_llm`函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79e5f128-8b02-40ca-b789-d577b6e38a3a",
   "metadata": {},
   "source": [
    "<table>\n",
    "  <thead>\n",
    "    <tr>\n",
    "      <th width = '80px'>角色</th>\n",
    "      <th width = '500px'>大模型算法工程师</th>\n",
    "      <th width = '500px'>小学老师</th>\n",
    "    </tr>\n",
    "  </thead>\n",
    "  <tbody valign = 'top'>\n",
    "    <tr>\n",
    "      <td>提示模板</td>\n",
    "      <td>\"你是资深的大模型算法工程师，请根据用户的问题进行解答\\n\"<br>\"---------------------\\n\"<br>\"{context_str}\\n\"<br>\"---------------------\\n\"<br>\"问题：{query_str}\\n。\"<br>\"回答：\"</td>\n",
    "      <td>\"你是一名小学老师，请根据用户的问题进行解答\\n\"<br>\"---------------------\\n\"<br>\"{context_str}\\n\"<br>\"---------------------\\n\"<br>\"问题：{query_str}\\n。\"<br>\"回答：\"</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "      <td>用户问题</td>\n",
    "      <td>Qwen-VL</td>\n",
    "      <td>Qwen-VL</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "      <td>输出回答</td>\n",
    "      <td>Qwen-VL是阿里巴巴云推出的一种多模态预训练模型，它结合了文本和图像处理能力，能够理解和生成与图像相关的文本描述。简单来说，Qwen-VL可以看作是一个能够“看图说话”的人工智能助手，它不仅能够理解图片的内容，还能根据图片内容生成相应的文字描述或回答相关问题。这种技术在多个领域都有广泛的应用前景，比如自动图像标注、视觉问答系统、基于内容的图像搜索等。对于新媒体运营而言，Qwen-VL这样的工具可以帮助提高内容创作效率，增强用户体验，为用户提供更加丰富多元的信息展示方式。</td>\n",
    "      <td>小朋友们，我们今天来认识一个新朋友，它的名字叫做Qwen-VL！<br>Qwen-VL就像是一个非常聪明的机器人，它拥有“眼睛”👀和“大脑”🧠。它能“看”图片、视频，还能理解它们的意思，就像我们一样！<br>比如，你给它看一张小猫玩毛线球的图片，它不仅能认出这是一只小猫🐱，还能告诉你它在玩毛线球🧶。 更厉害的是，你还可以问它：“小猫是什么颜色的？”，它就能告诉你小猫的颜色。<br>它还能理解视频里的内容，比如你给它看一段小朋友们踢足球⚽️的视频，它能告诉你谁在踢球，球踢到了哪里，甚至还能猜到接下来可能会发生什么！<br>Qwen-VL就像一个正在学习的小朋友，它还在不断地学习和进步，将来它会变得更聪明，能做更多的事情，帮助我们更好地了解这个世界！</td>\n",
    "    </tr>\n",
    "  </tbody>\n",
    "</table>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10f06bea-a34b-4e7a-b2b5-918af4c1ebdb",
   "metadata": {},
   "source": [
    "### 4.3 规定输出格式\n",
    "有时候，开发者在设计大模型应用时，需要采用结构化的数据作为下游系统的输入，才能完成整个应用的开发，但是一般大模型是输出连续的文本。不用担心，大模型有结构化输出的能力。你只需在提示词中指定输出的格式和要求，大模型有很大可能会输出结构化的内容。\n",
    "\n",
    "在下面的示例中，将基于3.2中的提示词模板，以答疑机器人的身份帮助公司开发的文档进行文档质量审查，并将结果以json的格式输出："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6692496-b6e4-4d75-aaa2-ef38b0c24f0c",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:49:49.007218Z",
     "start_time": "2025-01-15T06:49:46.080617Z"
    }
   },
   "outputs": [],
   "source": [
    "question_task= \"\"\"\n",
    "【任务要求】\n",
    "你将看到一句话或一段话。你需要审查这段话中有没有错别字。如果出现了错别字，你要指出错误，并给出解释。\n",
    " “的” 和 “地” 混淆不算错别字，没有错误\n",
    "---\n",
    "【输出要求】\n",
    "请你只输出json格式，不要输出代码段\n",
    "其中，label只能取0或1，0代表有错误，1代表没有错误\n",
    "reason是错误的原因\n",
    "correct是修正后的文档内容\n",
    "---\n",
    "【用户输入】\n",
    "以下是用户输入，请审阅：\n",
    "\"\"\"\n",
    "question_doc = \"分隔符是特殊的符号，它们帮助大语言模型 (LLM) 识别提示中哪些部分应当被视为一个完整的意思单元。\"\n",
    "\n",
    "question = question_task + question_doc\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebe59cd3-a241-4b28-902e-7ca35966e6a7",
   "metadata": {},
   "source": [
    "由上述示例的结果可知，在提示词`question_task`中注明了输出格式为json，也规定了输出的内容，大模型成功的输出了格式化的内容。这种稳定的格式化输出，使得在现有的系统中接入大模型这个操作变得具有可行性。\n",
    "\n",
    "在新闻网站、博客平台或企业内部的知识分享平台上，用户编辑或发布的文章可能会包含错别字、语法错误、甚至是敏感信息。但是传统的人工审核方式很容易出疏漏。这时候可以接入大模型来对内容做审查工作。如果文章被标记为存在严重语法错误或含有高风险敏感词汇，则将其修改的优先级设置为“高”。对于轻微问题的文章，则可以将其修改的优先级设置为“低”。这样会节省人力成本，提高系统的效率与准确性。\n",
    "\n",
    "当然了，类似于上述场景的应用非常多，开发者可以分析系统中流程的瓶颈或者关注数据密集型的任务，探索更多的大模型应用场景。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75f32b7a-9b47-40f1-82cb-e889a3b48048",
   "metadata": {},
   "source": [
    "### 4.4 提供少样本示例\n",
    "\n",
    "在4.3的例子中，提示词规定了输出的格式，大模型成功生成了格式化的内容。然而，如果希望大模型输出的内容不仅格式正确，而且风格和结构也保持一致，可以提供几个样例作为参考。这相当于给大模型提供了一本“参考书”。下面的代码示例中，先观察下没有样例时的大模型的输出吧！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afcca16d368baf38",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T07:20:42.883097Z",
     "start_time": "2025-01-15T07:20:18.423060Z"
    }
   },
   "outputs": [],
   "source": [
    "question_task = \"\"\"\n",
    "【任务要求】\n",
    "请你根据用户的主题，创作内容。\n",
    "---\n",
    "【输出要求】\n",
    "最终输出需要以Markdown格式呈现，请注意，在你的回答中包含所有必要的Markdown元素，如标题、列表、链接、图片引用、加粗等，以便于阅读、后续编辑和保存。\n",
    "---\n",
    "【用户输入】\n",
    "以下是用户的要求创作的主题：\n",
    "\"\"\"\n",
    "question_doc = \"手工钥匙扣制作教程\"\n",
    "\n",
    "question = question_task + question_doc\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30f61d316e005c4f",
   "metadata": {},
   "source": [
    "上述示例大模型成功了输出了手工钥匙扣制作教程，但是内容不够简洁，如果只想大模型输出特定风格和结构的内容，如只输出主题、材料清单、步骤等内容，可以为大模型添加几个样例，让它去“模仿”！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1e0f67f703e7ba",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:50:39.297041Z",
     "start_time": "2025-01-15T06:50:24.071562Z"
    }
   },
   "outputs": [],
   "source": [
    "question_task= \"\"\"\n",
    "【任务要求】\n",
    "请根据用户的主题，结合下面【样例】给的例子，理解和使用一致的风格和结构继续创作内容，不要输出多余的内容。\n",
    "---\n",
    "【输出要求】\n",
    "最终输出需要以Markdown格式呈现，请注意，在你的回答中包含所有必要的Markdown元素，如标题、列表、链接、图片引用、加粗等，以便于阅读、后续编辑和保存。\n",
    "---\n",
    "【样例】\n",
    "### 示例1: 制作简易书签\n",
    "# 简易书签制作教程\n",
    "\n",
    "## 材料清单\n",
    "- 彩色卡纸\n",
    "- 剪刀\n",
    "- 装饰贴纸\n",
    "- 铅笔\n",
    "\n",
    "## 步骤\n",
    "1. 选择一张彩色卡纸。\n",
    "2. 用铅笔在卡纸上画出一个长方形，尺寸约为2英寸 x 6英寸。\n",
    "3. 沿着铅笔线剪下长方形。\n",
    "4. 使用装饰贴纸对书签进行个性化装饰。\n",
    "5. 完成！现在你有了一个独一无二的书签。\n",
    "\n",
    "## 结束语\n",
    "希望这个教程能帮助你制作出满意的书签！\n",
    "\n",
    "---\n",
    "【用户输入】\n",
    "以下是用户的要求创作的主题：\n",
    "\"\"\"\n",
    "question_doc = \"制作手工贺卡\"\n",
    "\n",
    "question = question_task + question_doc\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e95447ee9772539d",
   "metadata": {},
   "source": [
    "由上述示例结果可知，大模型完全按照了样例输出了相同结构和风格的内容。在提示词中规定输出格式的同时，建议提供几个样例供大模型参考，这样可以使得大模型的输出更加稳定和一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53569b47-1619-4527-a9c8-b53f5e5bdf8d",
   "metadata": {},
   "source": [
    "### 4.5 给模型“思考”的时间\n",
    "\n",
    "对于一些复杂的任务来说，使用上面提到的提示词也许还不能帮助大模型完成任务。但是你可以通过让大模型一步步“思考”，引导大模型输出任务的中间步骤，允许大模型在进行推理之前，得到更多的依据，从而提升在复杂任务的表现能力。思维链（COT）方法是让模型进行思考的一种方法。它通过让模型处理中间步骤，逐步将复杂问题分解为子问题，最终推导出正确答案。\n",
    "\n",
    "假设有这样的场景，让大模型计算下面这道数学题，在此先提示一下，这道题的正确答案为10500元。先使用简单的提示词："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3569f469-c601-406a-a26d-fb0a74157c68",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:50:42.416991Z",
     "start_time": "2025-01-15T06:50:41.378053Z"
    }
   },
   "outputs": [],
   "source": [
    "question = \"\"\"\n",
    "【背景信息】\n",
    "某教育培训机构（以下简称“公司”）在2023年度发生了以下主要支出：\n",
    "为了给不同城市的学校学生上课，公司的老师全年共出差了5次，每次出差时间为一周，具体费用如下：\n",
    "   - 交通费及住宿费：平均1600元/次\n",
    "   - 教学用具采购费用：公司在年初一次性购买了一批教学用具，总价为10000元，预计可以使用4年。\n",
    "   \n",
    "【问题描述】\n",
    "请根据上述背景信息，完成以下任务：\n",
    "计算全年因教师出差而产生的差旅总费用，包括摊销的教学用具。\n",
    "\n",
    "【输出要求】\n",
    "直接给出总差旅费用，不要其他信息\"\"\"\n",
    "\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edf9078a-f716-4537-b003-a90d70f988b5",
   "metadata": {},
   "source": [
    "由上面的实验结果来看，大模型计算结果不正确。下面将使用COT方法，让大模型逐步进行思考。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92cb1715-305c-43ee-bd5e-274d2556ad56",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-15T06:50:55.556383Z",
     "start_time": "2025-01-15T06:50:43.720893Z"
    }
   },
   "outputs": [],
   "source": [
    "question = \"\"\"某教育培训机构（以下简称“公司”）在2023年度发生了以下主要支出：\n",
    "为了给不同城市的学校学生上课，公司的老师全年共出差了5次，每次出差时间为一周，具体费用如下：\n",
    "   - 交通费及住宿费：平均1600元/次\n",
    "   - 教学用具采购费用：公司在年初一次性购买了一批教学用具，总价为10000元，预计可以使用4年。\n",
    "   \n",
    "### 问题描述\n",
    "请根据上述背景信息，完成以下任务：\n",
    "计算全年因教师出差而产生的差旅总费用，包括摊销的教学用具。\n",
    "\n",
    "### 输出要求\n",
    "请你一步步推导，计算总差旅费用\"\"\"\n",
    "\n",
    "ask_llm(question, query_engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0ec9cdf-d6e1-469f-9e4c-f91fdf7f1c3e",
   "metadata": {},
   "source": [
    "经过优化后的提示词，大模型能够准确计算出结果。因此在开发大模型应用时，可以在提示词中添加思维链的方法，可以确保一些推理任务能正确执行。  \n",
    "\n",
    "使大模型进行 “思考”的方法还有很多种，比如：思维树（ToT)、思维图（GOT） 等。但是就目前大模型的发展来说，仅靠引导大模型“思考”还是无法完成更复杂的工作。大模型也逐渐从COT的提示方法向多智能体（Agent）方向进行发展。你可以在<2.6 用插件扩展答疑机器人的能力边界>阅读详细内容。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "n6dhhkohbt",
   "metadata": {},
   "source": [
    "### 4.6 Meta Prompting: 让大模型成为你的提示词教练"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "i11zcld9f3m",
   "metadata": {},
   "source": [
    "一次性就写好一个完美的提示词，往往非常困难。更常见的工作流是：\n",
    "1.  写出第一版提示词。\n",
    "2.  运行它，并分析输出结果中有哪些不符合预期的地方。\n",
    "3.  总结问题，思考如何改进，然后修改提示词。\n",
    "4.  不断重复这个迭代过程，直到满意为止。\n",
    "\n",
    "你可以回想一下，这是否也是你优化提示词的常规路径？这个过程虽然有效，但非常依赖经验，也颇为耗时。\n",
    "\n",
    "这时，你可能会想到：既然大模型这么强大，**这个分析、总结、改进的迭代过程，是否能让大模型自己来做呢？** 让它扮演“提示词评审专家”的角色，帮助我们分析和优化提示词，无疑会更高效。\n",
    "\n",
    "答案是肯定的。这种**让你和模型一起“讨论”如何优化提示词本身**的方法，就叫做 **Meta Prompting**。\n",
    "\n",
    "为了进一步理解这个强大的技巧，你可以来亲手实践一下。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7jtomn45xs",
   "metadata": {},
   "source": [
    "#### 步骤 1：一个不甚理想的初始提示词\n",
    "\n",
    "假设你的任务是优化答疑机器人的回答，让它在回答新员工关于“公司福利”的问题时，输出更友好、结构更清晰的内容。你可能会从一个简单的提示词开始："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "qc1fi30l5mc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 在真实的 RAG 应用中，这段文本会由你的向量数据库检索而来。\n",
    "# 这里用一个字符串来模拟它，方便你进行实验。\n",
    "retrieved_text = \"\"\"\n",
    "关于公司的福利政策，我们提供全面的健康保险，覆盖员工及其直系家属。\n",
    "年度体检是标配。此外，每年有15天的带薪年假，以及5天的带薪病假。\n",
    "我们还提供每月500元的交通补贴和300元的餐饮补贴。\n",
    "为了鼓励员工成长，公司设有每年高达8000元的教育培训基金，员工可以申请用于课程学习或购买专业书籍。\n",
    "健身方面，公司与多家健身房有合作，员工可享受折扣价。\n",
    "\"\"\"\n",
    "\n",
    "# 这是一个非常基础的提示词，只是简单地将任务和信息拼接起来。\n",
    "initial_prompt = f\"\"\"\n",
    "根据以下信息，回答新员工关于公司福利的问题。\n",
    "\n",
    "【参考信息】\n",
    "{retrieved_text}\n",
    "\"\"\"\n",
    "\n",
    "# 看看这个“朴素”的提示词会产生什么样的效果。\n",
    "response = llm.invoke(initial_prompt)\n",
    "print(\"--- 初始回答 ---\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l9df0nw4kwc",
   "metadata": {},
   "source": [
    "这个回答虽然包含了所有信息，但你可能会觉得，对于一个刚入职的新员工来说，它显得有些平淡和杂乱。它只是简单地复述了文本，没有重点，也缺乏热情的欢迎语气。\n",
    "\n",
    "显然，这个效果没有达到你的预期。现在，你无需自己苦思冥想如何修改，而是可以尝试一种更高效的方法：**让大模型来帮你优化**。\n",
    "\n",
    "#### 步骤 2：构建 Meta Prompt，让大模型提供优化建议\n",
    "\n",
    "现在，你对这个平淡的回答不满意。你可以构建一个“Meta Prompt”，清晰地向模型描述你的目标（友好、结构化、重点突出），并将你那不甚理想的初始提示词和它产出的回答一并“喂”给大模型，请求它以“提示词工程专家”的身份，帮你改进。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "mthl0mpq7u",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 你需要将你的不满和期望清晰地表达出来，这是让AI教练理解你意图的关键。\n",
    "meta_prompt = f\"\"\"\n",
    "我正在为公司的新员工答疑机器人优化一个提示词，目标是回答关于“公司福利”的问题。\n",
    "\n",
    "这是我的第一个尝试：\n",
    "---\n",
    "{initial_prompt}\n",
    "---\n",
    "\n",
    "这是它生成的输出：\n",
    "---\n",
    "{response}\n",
    "---\n",
    "\n",
    "这个输出不够好。我希望机器人的回答更具吸引力，并且结构清晰，能让新员工快速抓住重点。具体要求如下：\n",
    "1.  **语气**：友好、热情，有欢迎新同事的感觉。\n",
    "2.  **结构**：使用清晰的要点（比如用表情符号开头的列表）来组织内容。\n",
    "3.  **内容**：将福利分为几个类别，如“健康与假期”、“补贴与激励”等。\n",
    "\n",
    "请你扮演一位提示词工程专家，帮我重写这个提示词，以实现上述目标。\n",
    "\"\"\"\n",
    "\n",
    "# 现在，让AI教练开始工作，为你生成一个优化版的提示词。\n",
    "optimization_suggestion = llm.invoke(meta_prompt)\n",
    "print(\"--- 来自AI教练的优化建议 --\")\n",
    "print(optimization_suggestion)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "o856e9fcof",
   "metadata": {},
   "source": [
    "观察AI教练给出的建议，你会发现，它给出的优化后提示词，很可能应用了你在前面章节学到的多种技巧，例如：\n",
    "\n",
    "*   **明确角色** (例如，“你是一位热情、友好的入职伙伴”)\n",
    "*   **清晰的任务描述** (例如，“根据提供的参考信息，生成一段关于公司福利的介绍”)\n",
    "*   **规定输出格式和风格** (例如，“使用热情的欢迎语”、“用表情符号开头的列表”)\n",
    "\n",
    "这证明，让模型扮演专家来优化提示词是完全可行的。\n",
    "\n",
    "#### 步骤 3：使用优化后的提示词\n",
    "\n",
    "现在，你可以直接将这位“AI教练”为你量身定做的提示词用于你的任务，看看效果如何。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21fwswiytnc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这是一个假设的、由AI教练建议的优化版提示词\n",
    "# 在实际应用中，你可以直接使用 `optimization_suggestion` 的输出\n",
    "# 这里为了演示，我们手动构建一个符合建议的提示词\n",
    "\n",
    "optimized_prompt = f\"\"\"\n",
    "【角色】\n",
    "你是一位热情、友好的入职伙伴（Onboarding Buddy），你的任务是欢迎新同事，并清晰地介绍公司的福利政策。\n",
    "\n",
    "【任务】\n",
    "根据提供的【参考信息】，生成一段关于公司福利的介绍。\n",
    "\n",
    "【输出要求】\n",
    "1.  开头使用热情洋溢的欢迎语。\n",
    "2.  将福利信息分类整理，例如分为“健康与假期”、“补贴与激励”等。\n",
    "3.  每个福利项目前使用一个相关的表情符号，使其更生动。\n",
    "4.  结尾表达对新同事的美好祝愿。\n",
    "\n",
    "【参考信息】\n",
    "{retrieved_text}\n",
    "\"\"\"\n",
    "\n",
    "# 使用优化后的提示词再次调用模型\n",
    "final_response = llm.invoke(optimized_prompt)\n",
    "print(\"--- 使用优化版提示词的回答 ---\")\n",
    "print(final_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pdlgh7ddj5q",
   "metadata": {},
   "source": [
    "通过这个迭代过程，你将得到一个语气热情、结构清晰的回答，这无疑会给新员工留下更好的第一印象。\n",
    "\n",
    "这个例子展示了 Meta Prompting 在提升用户体验方面的巨大价值。它不仅仅是提取信息，更是关于**如何更好地呈现信息**。当你遇到一个棘手的提示词难题时，不要忘记，你的模型本身就是最好的教练。通过清晰地描述你的目标和困难，你可以引导它为你构建出更强大、更精确、也更有“人情味”的提示词。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "oqor78pg46l",
   "metadata": {},
   "source": [
    "在刚才的初步方案中，我们是向“AI教练”描述了我们期望的**定性目标**（如“友好”、“结构更清晰”），然后由它直接为我们生成一个优化后的提示词。这个方法很便捷，但它有一个关键的局限性：AI教练对这些定性目标的理解可能不够精确，你描述的需求也可能不够具体，这导致优化结果的好坏存在不确定性。\n",
    "\n",
    "为了解决这个问题，让优化过程变得更可控、更精确，我们需要从“定性指导”升级到“量化对齐”。与其给出一个模糊的目标，不如直接给出一个完美的 **“参考答案”** 作为精确的靶子。接下来的进阶方案将向你展示如何利用这个“参考答案”，让大模型通过自动化的差距分析和迭代，逐步地、精确地向着最优结果逼近。这才是更具工程化的改进思路。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "r5lli1fdexj",
   "metadata": {},
   "source": [
    "#### 多轮迭代：引入参考答案进行差距分析\n",
    "\n",
    "在前面的例子中，你扮演了主导角色，接收“AI教练”的建议并手动应用它。但这个过程还可以进一步自动化和精确化。与其让评估者给出一个模糊的“好”或“不好”的判断，一个更高级的方法是引入一个 **“参考答案”（Reference Answer）**。\n",
    "\n",
    "这个“参考答案”是你心目中最完美的理想答案，可以由人类专家撰写，也可以用一个非常详尽的提示词让最强大的模型生成。迭代优化的目标就变成了：**不断修改提示词，使其生成的回答与这个“参考答案”之间的差距越来越小**。\n",
    "\n",
    "这个过程就像一个拥有精确制导系统的自我修正流程：\n",
    "\n",
    "1.  **设定参考答案 (Set Reference Answer)**：首先，定义一个高质量的、理想的“参考答案”。\n",
    "2.  **生成 (Generate)**：使用当前待优化的提示词，生成一个回答。\n",
    "3.  **分析差距 (Analyze Gap)**：让一个“评估者”大模型（Critic）来比较“生成的回答”和“参考答案”，并输出一份详细的“差距分析报告”，指出两者在语气、结构、内容、格式等方面的具体差异。\n",
    "4.  **优化 (Optimize)**：将“差距分析报告”连同原始提示词、生成的回答一起，交给一个“优化者”大模型（Optimizer）。它的任务是根据这份报告，重写提示词，以专门解决其中指出的问题，从而缩小差距。\n",
    "5.  **重复 (Repeat)**：用优化后的新提示词替换旧的，然后回到第2步，直到“评估者”认为两者差距足够小，或达到最大迭代次数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "kda6wboa3z",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. 设定参考答案\n",
    "reference_answer = \"\"\"\n",
    "👋 欢迎加入我们的大家庭！很高兴能为你介绍我们超棒的福利政策：\n",
    "\n",
    "**🏥 健康与假期，我们为你保驾护航：**\n",
    "-   **全面健康保险**：覆盖你和你的家人，安心工作无烦忧。\n",
    "-   **年度体检**：你的健康，我们时刻关心。\n",
    "-   **带薪年假**：每年足足15天，去探索诗和远方吧！\n",
    "-   **带薪病假**：5天时间，让你安心休养，快速恢复活力。\n",
    "\n",
    "**💰 补贴与激励，为你加油打气：**\n",
    "-   **交通补贴**：每月500元，通勤路上更轻松。\n",
    "-   **餐饮补贴**：每月300元，午餐加个鸡腿！\n",
    "-   **教育培训基金**：每年高达8000元，投资自己，未来可期。\n",
    "-   **健身折扣**：与多家健身房合作，工作再忙也别忘了锻炼哦！\n",
    "\n",
    "希望这些福利能让你感受到公司的关怀！期待与你一起创造更多价值！🎉\n",
    "\"\"\"\n",
    "\n",
    "# 2. 定义差距分析和优化函数\n",
    "def analyze_gap(generated_response, reference):\n",
    "    gap_analysis_prompt = f\"\"\"\n",
    "    【角色】你是一位文本比较专家。\n",
    "    【任务】请详细比较【生成回答】与【参考答案】之间的差距。\n",
    "    【参考答案】\n",
    "    {reference}\n",
    "    ---\n",
    "    【生成回答】\n",
    "    {generated_response}\n",
    "    ---\n",
    "    【要求】\n",
    "    请从语气、结构、内容细节、格式（如表情符号使用）等方面，输出一份详细的差距分析报告。如果两者几乎没有差距，请直接回答“差距很小”。\n",
    "    \"\"\"\n",
    "    return llm.invoke(gap_analysis_prompt)\n",
    "\n",
    "def optimize_prompt_with_gap_analysis(current_prompt, generated_response, gap_report):\n",
    "    optimization_prompt = f\"\"\"\n",
    "    【角色】你是一位顶级的提示词工程师。\n",
    "    【任务】根据提供的“差距分析报告”，优化“当前提示词”，使其能够生成更接近“参考答案”的输出。\n",
    "    ---\n",
    "    【当前提示词】\n",
    "    {current_prompt}\n",
    "    ---\n",
    "    【它生成的回答】\n",
    "    {generated_response}\n",
    "    ---\n",
    "    【差距分析报告】\n",
    "    {gap_report}\n",
    "    ---\n",
    "    【要求】\n",
    "    请只返回优化后的新提示词，不要包含任何其他解释。\n",
    "    \"\"\"\n",
    "    return llm.invoke(optimization_prompt)\n",
    "\n",
    "# 3. 迭代优化循环\n",
    "current_prompt = initial_prompt # 复用4.6节的initial_prompt\n",
    "for i in range(3): # 最多迭代3次\n",
    "    print(f\"--- 第 {i+1} 轮迭代 ---\")\n",
    "    generated_response = llm.invoke(current_prompt.format(retrieved_text=retrieved_text))\n",
    "    print(f\"生成的回答 (部分):\\\\n{generated_response[:100]}...\")\n",
    "    \n",
    "    gap_report = analyze_gap(generated_response, reference_answer)\n",
    "    print(f\"差距分析报告:\\\\n{gap_report}\")\n",
    "    \n",
    "    if \"差距很小\" in gap_report:\n",
    "        print(\"\\\\n评估通过，优化完成！\")\n",
    "        break\n",
    "    \n",
    "    print(\"\\\\n评估未通过，根据差距分析报告优化提示词...\")\n",
    "    current_prompt = optimize_prompt_with_gap_analysis(current_prompt, generated_response, gap_report)\n",
    "else:\n",
    "    print(\"\\\\n达到最大迭代次数，停止优化。\")\n",
    "\n",
    "final_prompt_based_on_reference = current_prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "zeaxehmn60s",
   "metadata": {},
   "source": [
    "这个自动化的迭代过程展示了 Meta Prompting 的真正效果。它将你从繁琐的手动调整中解放出来，让模型自己去探索和发现最佳的表达方式，也为更复杂的 **AI 自我改进（Self-Improvment）** 系统的设计提供了基础思路。\n",
    "\n",
    "#### 效果评测：量化你的优化成果\n",
    "\n",
    "当你有多个版本的提示词（例如，初始版本 vs. 单次优化版 vs. 多轮迭代最终版），你如何客观地证明哪个更好呢？除了直观感受，更科学的方法是进行**量化评估**。\n",
    "\n",
    "你可以再次利用大模型，让它扮演一个“评分员”（Grader）的角色，根据一系列标准对不同提示词生成的回答进行打分。\n",
    "\n",
    "例如，你可以定义一个评分标准：\n",
    "\n",
    "*   **友好度 (Friendliness)**: 1-5分\n",
    "*   **结构清晰度 (Clarity)**: 1-5分\n",
    "*   **信息准确性 (Accuracy)**: 1-5分\n",
    "\n",
    "然后，构建一个“Grader Prompt”，将评分标准和待评估的回答一起交给大模型，让它输出一个结构化的评分结果（如 JSON）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "qho0sd1rgc8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAJOCAYAAACqS2TfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB4YElEQVR4nO3deXgN5///8ddJyEJWEbWL2PedoraqvVqUoFVLUYoWrU/bVEtiKW1VN1W0iLZaWylVuwpqV0ttsQZRO5HYEiTz+8Mv5+vIIomMbM/HdeVqzz33zLwnmXHO68w9MxbDMAwBAAAAAIA0Z5feBQAAAAAAkFURugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQDIhIKCgmSxWGSxWHTy5Mn0LidNnDx50rpNQUFB6V0OAABpgtANAMhWgoODrcEuuT9DhgxJ77IBAEAmRegGAACmivvyIiAgIL1LAQDgicuR3gUAAJBe3njjDQ0YMOCR/fLmzfsEqoGPj48Mw0jvMgAASFOEbgBAtpUvXz5VrFgxvcsAAABZGMPLAQAAAAAwCaEbAIBk+vHHH63XJ69evfqR/fv16yeLxSJHR0eFh4fbTNu/f7/GjBmjFi1aqHDhwnJ0dJSLi4tKlSqlHj16aOvWrY9Vq4+PjywWi3r27Jlkv549e8piscjHxyfB6eHh4Zo5c6a6deum8uXLy8XFRQ4ODsqfP79atGihadOm6c6dO0nWECcwMDDeTeoerC+5dy+/c+eOJk+erCZNmsjb29taT+vWrfXzzz8rNjY22dt77do1jRgxQhUqVFDu3Lnl4eGhhg0bavbs2YkuAwCAlGB4OQAAydS+fXv1799ft2/f1i+//KJmzZol2vfu3btasGCBJKl169by9PS0TgsODlaTJk3izXPnzh0dO3ZMx44d048//qj3339f48aNS/sNSYFq1arp1KlT8dovXLigVatWadWqVZoyZYqWLVum/Pnzm17PyZMn1apVK4WEhMSrZ/ny5Vq+fLmmTp2qxYsXK0+ePEku6/Dhw2rZsmW8R65t3LhRGzdu1JYtWzRp0qS03gQAQDbDmW4AAJLJ1dVVL7zwgiRp4cKFioqKSrTv8uXLdfXqVUnSK6+8YjPt3r17yp07t/z8/DRlyhQFBwdr165dWrFihT7//HMVK1ZMkjR+/HjNnDnTpK1JnpiYGNWpU0ejR4/W0qVLtWPHDm3atEk///yzWrZsKUnavXu3unTpEm/eVatWad++fdbXb7zxhvbt22fzM3bs2GTXcuPGDTVt2tQauNu1a6clS5Zo586dmj9/vho1aiRJ+vvvv9W2bVvFxMQkuqxbt26pbdu2unLlij788EMFBwdr586d+v7771W4cGFJ0rfffquVK1cmuz4AABLCmW4AQLZ18eJF7d+//5H9ypQpo5w5c0q6H6Dnzp2ryMhILV26VB07dkxwnl9++UWS5Obmpueff95mWtWqVXXmzBl5eHjEm69FixYaNGiQnn/+ea1evVqBgYHq3r277O3tU7h1aeOvv/5SqVKl4rXXq1dPr7zyimbOnKnXXntN69ev19q1a9W0aVNrn9KlS9vM87g3rgsMDNSJEyckSR9++KFGjx5tnVajRg299NJLevXVVzV79mxt3rxZ06ZN0xtvvJHgsi5duqQ7d+5oy5YtqlChgs1yGjdurEqVKikqKkqTJ09WixYtUl0zAACc6QYAZFvfffedKlWq9Mif//77zzpPy5Yt5eXlJUmJXvd748YNLVmyRJL00ksvycnJyWZ63rx5EwzccRwcHPTZZ59Jkk6dOqU9e/Y8xlY+noQC94N69eqlqlWrSpJ+//130+qIjo7WDz/8IEmqUKFCgs/8tlgsmjx5svXv86ih4aNHj7YJ3HFKliypdu3aSbp/1hwAgMdB6AYAIAVy5sypTp06Sbo/hPzatWvx+ixatEi3b9+WFH9oeUKio6N1+vRpHTx4UPv379f+/fttnle9d+/etCn+MRmGofPnz+vIkSPWOvfv369ChQpJMrfOf/75x/q77tmzZ6Jn/t3c3OTn5ydJOnjwoM6dO5dgP4vFopdffjnR9dWoUUOSdPXq1QT/xgAAJBehGwCQbY0cOVKGYTzy5+E7e8cF6ejoaOvN0h4UN7S8YMGCCd4wTZJu3rypcePGqUqVKsqdO7eKFSumChUqWM+uV6tWzdr38uXLabTFqfPnn3/q+eefl7u7uwoUKKAyZcrYjAT4888/Ta/zwcsA6tSpk2TfB6cndvlA3rx5rWfEE/LgTdiuX7+e3DIBAIiHa7oBAEih+vXrq1ixYjp16pRmz56tPn36WKddvHhRa9askSR16dJFdnbxv98+efKknn32WYWGhiZrfXFnzZ80wzDUt29fTZ8+PVn9zawz7qZ00v1rw5Py4F3UH5zvQbly5UpyGQ/+3ZK6IRsAAI/CmW4AAFLowaHJGzZssLnme968ebp3756kxIeWv/rqqwoNDZXFYtFrr72mVatWKSwsTFFRUYqNjZVhGDZB78Gh5k/SjBkzrIG7atWqCgoK0qFDhxQZGal79+5ZRwK8+uqrT7TOB5/9DQBARkfoBgAgFeICdWxsrH799Vdre9zQ8rJly6p69erx5gsJCbHenOuDDz7Q9OnT1axZMxUuXFiOjo7WQJnYGdrkijtTGxsbm2S/mzdvJjrt+++/l3T/xmKbN29Wjx49VLZsWbm6utpcU/24tSbHg8O9L1y4kGTf8+fPJzgfAADpgdANAEAqVKhQQVWqVJH0f0E7NDRUW7ZskZT4We4DBw5Y/79z586JLn/nzp2PVZ+rq6skKTw8PMl+R44cSXRaXK0vvPCCnJ2dE+xjGIZ27dqVyiqT78FHjW3bti3Jvtu3b09wPgAA0gOhGwCAVIoL1rt379ahQ4es4VtSonfGjht6LiV9lnnKlCmPVVvx4sUlSbt27Up02PeBAwf077//JrqMuFqTqnPx4sWJ3iE8Ttwj06Kjo5Psl5QaNWpYH7M2a9asRM/gX79+XfPmzZMklS9fXgUKFEj1OgEASAuEbgAAUqlr167W4eCzZ8+2DjOvW7eufH19E5znwedeBwUFJdjnu+++0+LFix+rtkaNGkmSzp49azP8Pc7169fVu3fvJJcRV+sff/yR4BDy48ePa+DAgY+sJS74Hj9+/JF9E+Po6Gi9Yd3+/fs1evToeH0Mw9CgQYOsd1EfNGhQqtcHAEBa4e7lAIBs6+LFi4k+UupBzs7OKlGiRLz2woULq1GjRgoODta3335rfZ5zUs/mrlatmipWrKj9+/dr6tSpCg8P16uvvqoCBQrozJkz+vnnn7VgwQLVr19fmzZtSvW2devWTQEBAYqMjFTv3r117NgxtWjRQhaLRf/8848mTpyoM2fOqFq1atq9e3eCy+jevbv+97//6ezZs6pbt67ee+89VaxYUVFRUfrrr7/05ZdfKjo6WtWrV09yiHm9evUUGhqqJUuWaOrUqapfv7717Lebm9sj70YeZ8SIEVq4cKFOnDihgIAA7du3T7169VKBAgUUGhqqSZMmKTg4WNL9Lz5ef/31lP3SAAAwgwEAQDaybt06Q1KKfqpUqZLo8r7//nubvjly5DAuXryYZA27d+82PD09E11fpUqVjLNnz1pfjxw5Mt4yZs6caZ0eGhqa4HrmzZtn2NvbJ7gOZ2dnY/78+UaPHj0MSUaxYsXizX/nzh2jefPmidbp7OxszJs3L8llxG2vo6Njgsvo0aOHtV9oaKi1febMmQkuKzQ01ChbtmySf6/69esbV65cSXD+R9UaJzm/XwAAkoPh5QAAPIaOHTvK0dHR+rp58+by9vZOcp6qVatqz5496t+/v4oVK6acOXMqT548ql27tiZMmKDt27enybXInTp10ubNm9W+fXt5e3vLwcFBRYoUUY8ePbRjxw517Ngxyflz5sypP//8U19//bVq1qypXLlyydnZWSVLllT//v21a9cuderU6ZF1VK1aVVu2bFHXrl1VtGhRm99XSvn4+Gjv3r2aNGmSGjVqJC8vL+XMmVNPPfWUWrZsqZ9++kkbNmzgruUAgAzDYhjp9PBPAAAAAACyOM50AwAAAABgEkI3AAAAAAAmIXQDAAAAAGCSDBe6//vvP3Xr1k1eXl5ydnZWpUqVtHPnzvQuCwAAAACAFMtQz+kODw9X/fr11aRJEy1fvlze3t46evSoPD0907s0AAAAAABSLEPdvfz999/Xpk2btHHjxvQuBQAAAACAx5ahQnf58uXVokULnTlzRuvXr1ehQoU0YMAA9e3bN8H+0dHRio6Otr6OjY3V1atX5eXlJYvF8qTKBgAAAABkM4Zh6Pr16ypYsKDs7BK/cjtDhW4nJydJ0ttvv61OnTppx44dGjx4sKZMmaIePXrE6x8QEKDAwMAnXSYAAAAAAJKksLAwFS5cONHpGSp0Ozg4qGbNmtq8ebO17a233tKOHTu0ZcuWeP0fPtMdERGhokWLKiwsTG5ubk+kZgAAAABA9hMZGakiRYro2rVrcnd3T7RfhrqRWoECBVS+fHmbtnLlyum3335LsL+jo6McHR3jtbu5uRG6AQAAAACme9SlzRnqkWH169fX4cOHbdqOHDmiYsWKpVNFAAAAAACkXoYK3UOHDtXWrVv18ccf69ixY/rll180bdo0DRw4ML1LAwAAAAAgxTJU6K5Vq5YWLVqkX3/9VRUrVtTo0aP15Zdf6pVXXknv0gAAAAAASLEMdSO1xxUZGSl3d3dFRERwTTcApJGYmBjdvXs3vcsAkAZy5swpe3v79C4DALKE5ObPDHUjNQBAxmEYhs6fP69r166ldykA0pCHh4fy58//yBv/AADSBqEbAJCguMCdL18+5cqViw/oQCZnGIZu3bqlixcvSrr/1BgAgPkI3QCAeGJiYqyB28vLK73LAZBGnJ2dJUkXL15Uvnz5GGoOAE9AhrqRGgAgY4i7hjtXrlzpXAmAtBZ3XHOvBgB4MgjdAIBEMaQcyHo4rgHgySJ0AwAAAABgEkI3AAAPOHnypCwWi/bs2ZPseYKCguTh4ZHudQAAgIyHG6kBAFKkeedRT3R9q+aOSPE8YWFhGjlypFasWKHLly+rQIECateunUaMGPHIG8MVKVJE586dU968eZO9vs6dO6t169YprjOt/Prrr+rWrZv69++vb7/9Nt3qAAAA8XGmGwCQpZw4cUI1a9bU0aNH9euvv+rYsWOaMmWK1q5dq7p16+rq1auJznvnzh3Z29srf/78ypEj+d9LOzs7K1++fGlRfqpMnz5d7777rn799VdFRUWlWx2pcefOnfQuAQAAUxG6AQBZysCBA+Xg4KBVq1apUaNGKlq0qFq1aqU1a9bov//+0/Dhw619fXx8NHr0aHXv3l1ubm56/fXXExzWvWTJEpUqVUpOTk5q0qSJZs2aJYvFomvXrkmKP7w8ICBAVatW1U8//SQfHx+5u7urS5cuun79urXPihUr9Mwzz8jDw0NeXl56/vnndfz48RRvb2hoqDZv3qz3339fpUuX1sKFC22mx9W2cuVKlStXTi4uLmrZsqXOnTtn7RMcHKzatWsrd+7c8vDwUP369XXq1ClFRETI3t5eO3fulCTFxsYqT548evrpp63z/vzzzypSpIj1dVhYmPz8/OTh4aE8efLoxRdf1MmTJ63Te/bsqXbt2mns2LEqWLCgypQpk+JtBgAgMyF0AwCyjKtXr2rlypUaMGCA9XnEcfLnz69XXnlFc+fOlWEY1vYJEyaoSpUq2r17tz766KN4ywwNDVXHjh3Vrl077d27V/369bMJ7ok5fvy4fv/9dy1dulRLly7V+vXrNX78eOv0mzdv6u2339bOnTu1du1a2dnZqX379oqNjU3RNs+cOVNt2rSRu7u7unXrpunTp8frc+vWLU2YMEE//fSTNmzYoNOnT2vYsGGSpHv37qldu3Zq1KiR/v33X23ZskWvv/66LBaL3N3dVbVqVQUHB0uS9u3bJ4vFot27d+vGjRuSpPXr16tRo0aS7j+CqkWLFnJ1ddXGjRu1adMma8h/8Iz22rVrdfjwYa1evVpLly5N0fYCAJDZcE03ACDLOHr0qAzDULly5RKcXq5cOYWHh+vSpUvW4eDPPvus3nnnHWufB8/KStLUqVNVpkwZffbZZ5KkMmXKaP/+/Ro7dmyStcTGxiooKEiurq6SpFdffVVr1661zvfSSy/Z9J8xY4a8vb118OBBVaxYMVnbG7eOb775RpLUpUsXvfPOOwoNDVXx4sWt/e7evaspU6aoRIkSkqRBgwZp1Kj71+ZHRkYqIiJCzz//vHX6g7+/xo0bKzg4WMOGDVNwcLCaNWumkJAQ/f3332rZsqWCg4P17rvvSpLmzp2r2NhY/fDDD9bHUs2cOVMeHh4KDg5W8+bNJUm5c+fWDz/8IAcHh2RtJwAAmRlnugEAWc6DZ7IfpWbNmklOP3z4sGrVqmXTVrt27Ucu18fHxxq4JalAgQK6ePGi9fXRo0fVtWtX+fr6ys3NTT4+PpKk06dPJ7v21atX6+bNm9abuOXNm1fNmjXTjBkzbPrlypXLGqgfriVPnjzq2bOnWrRoobZt2+qrr76yGXreqFEj/f3334qJidH69evVuHFjaxA/e/asjh07psaNG0uS9u7dq2PHjsnV1VUuLi5ycXFRnjx5FBUVZTN0vlKlSgRuAEC2QegGAGQZJUuWlMVi0aFDhxKcfujQIXl6esrb29valjt3blNqyZkzp81ri8ViM3S8bdu2unr1qr7//ntt27ZN27Ztk5SyG4tNnz5dV69elbOzs3LkyKEcOXJo2bJlmjVrls26EqrlwS8mZs6cqS1btqhevXqaO3euSpcura1bt0qSGjZsqOvXr2vXrl3asGGDTehev369ChYsqFKlSkmSbty4oRo1amjPnj02P0eOHNHLL79sXZ9Zv3MAADIiQjcAIMvw8vJSs2bNNHnyZN2+fdtm2vnz5zV79mx17tzZOvQ5OcqUKWO9kVicHTt2PFadV65c0eHDh/Xhhx+qadOm1mHvKV3G4sWLNWfOHJuAu3v3boWHh2vVqlUpWl61atXk7++vzZs3q2LFivrll18kSR4eHqpcubImTZqknDlzqmzZsmrYsKF2796tpUuXWq/nlqTq1avr6NGjypcvn0qWLGnz4+7unqJ6AADIKgjdAIAsZdKkSYqOjlaLFi20YcMGhYWFacWKFWrWrJkKFSr0yGuxH9avXz+FhITovffe05EjRzRv3jwFBQVJUorC+4M8PT3l5eWladOm6dixY/rrr7/09ttvp2gZP/30k7y8vOTn56eKFStaf6pUqaLWrVsneEO1hISGhsrf319btmzRqVOntGrVKh09ejTedd2zZ8+2Buw8efKoXLlymjt3rk3ofuWVV5Q3b169+OKL2rhxo0JDQxUcHKy33npLZ86cSdH2AQCQVRC6AQBZSqlSpbRz5075+vrKz89PJUqU0Ouvv64mTZpoy5YtypMnT4qWV7x4cS1YsEALFy5U5cqV9d1331nvXu7o6JiqGu3s7DRnzhz9888/qlixooYOHWq9UVtyzZgxQ+3bt08w+L/00ktasmSJLl++/Mjl5MqVSyEhIXrppZdUunRpvf766xo4cKD69etn7dOoUSPFxMRYr92W7gfxh9ty5cqlDRs2qGjRourQoYPKlSun3r17KyoqSm5ubinaPgAAsgqLkZK7zWRwkZGRcnd3V0REBG/uAPAYoqKirHfAdnJySu9yMpyxY8dqypQpCgsLS+9SgBTj+AaAtJHc/MkjwwAAeITJkyerVq1a8vLy0qZNm/TZZ59p0KBB6V0WAADIBAjdAAA8wtGjRzVmzBhdvXpVRYsW1TvvvCN/f//0LgsAAGQChG4AAB7hiy++0BdffJHeZQAAgEyIG6kBAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDALIdi8Wi33//PU2XGRAQoKpVq6bpMjMKHx8fffnll+ldxmM7efKkLBaL9uzZ81jLady4sYYMGZImNQEAsj6e0w0ASJGqYwKe6Pr2fJiy9V26dEkjRozQn3/+qQsXLsjT01NVqlTRiBEjVL9+fUnSuXPn5OnpaUK1j6dnz566du2a9QuBxo0bq2rVqk8s8AYFBWnIkCG6du2aTfuOHTuUO3du09cfExOjr7/+WjNmzNDRo0fl7Oysp59+Wh9++KH1b5dcD/8uJalIkSI6d+6c8ubN+1h1Lly4UDlz5nysZQAAsg9CNwAgS3nppZd0584dzZo1S76+vrpw4YLWrl2rK1euWPvkz58/HSt88u7cuSMHB4dUz+/t7Z2G1STMMAx16dJFa9as0WeffaamTZsqMjJS3377rRo3bqz58+erXbt2j7UOe3v7NPnb58mT57GXAQDIPhheDgDIMq5du6aNGzfqk08+UZMmTVSsWDHVrl1b/v7+euGFF6z9HhxeHjfkeOHChWrSpIly5cqlKlWqaMuWLTbL/v7771WkSBHlypVL7du318SJE+Xh4ZFkPT/88IPKlSsnJycnlS1bVpMnT072tvTs2VPr16/XV199JYvFIovFopMnT0qS9u/fr1atWsnFxUVPPfWUXn31VV2+fNk6b+PGjTVo0CANGTJEefPmVYsWLSRJEydOVKVKlZQ7d24VKVJEAwYM0I0bNyRJwcHB6tWrlyIiIqzrCwgIkBR/ePnp06f14osvysXFRW5ubvLz89OFCxes0+OG2v/000/y8fGRu7u7unTpouvXrye6vfPmzdOCBQv0448/qk+fPipevLiqVKmiadOm6YUXXlCfPn108+ZNm+VPnTrV+jfx8/NTRESEdfqsWbO0ePFi67YEBwfHG14eHBwsi8WilStXqlq1anJ2dtazzz6rixcvavny5SpXrpzc3Nz08ssv69atWza/37jh5XHLePinZ8+e1v6LFy9W9erV5eTkJF9fXwUGBurevXvJ3hcAAJkboRsAkGW4uLjIxcVFv//+u6Kjo1M07/DhwzVs2DDt2bNHpUuXVteuXa3BaNOmTerfv78GDx6sPXv2qFmzZho7dmySy5s9e7ZGjBihsWPH6tChQ/r444/10UcfadasWcmq56uvvlLdunXVt29fnTt3TufOnVORIkV07do1Pfvss6pWrZp27typFStW6MKFC/Lz87OZf9asWXJwcNCmTZs0ZcoUSZKdnZ2+/vprHThwQLNmzdJff/2ld999V5JUr149ffnll3Jzc7Oub9iwYfHqio2N1YsvvqirV69q/fr1Wr16tU6cOKHOnTvb9Dt+/Lh+//13LV26VEuXLtX69es1fvz4RLf3l19+UenSpdW2bdt409555x1duXJFq1evtrYdO3ZM8+bN0x9//KEVK1Zo9+7dGjBggCRp2LBh8vPzU8uWLa3bUq9evUTXHRAQoEmTJmnz5s0KCwuTn5+fvvzyS/3yyy/6888/tWrVKn3zzTcJzluvXj3rOs6dO6e//vpLTk5OatiwoSRp48aN6t69uwYPHqyDBw9q6tSpCgoKeuT+AwDIOhheDgDIMnLkyKGgoCD17dtXU6ZMUfXq1dWoUSN16dJFlStXTnLeYcOGqU2bNpKkwMBAVahQQceOHVPZsmX1zTffqFWrVtYQWrp0aW3evFlLly5NdHkjR47U559/rg4dOkiSihcvbg1dPXr0eOS2uLu7y8HBQbly5bIZEj1p0iRVq1ZNH3/8sbVtxowZKlKkiI4cOaLSpUtLkkqVKqVPP/3UZpkP3vzLx8dHY8aMUf/+/TV58mQ5ODjI3d1dFoslySHYa9eu1b59+xQaGqoiRYpIkn788UdVqFBBO3bsUK1atSTdD+dBQUFydXWVJL366qtau3ZtomHzyJEjKleuXILT4tqPHDlibYuKitKPP/6oQoUKSZK++eYbtWnTRp9//rny588vZ2dnRUdHJ2s4+ZgxY6zXjPfu3Vv+/v46fvy4fH19JUkdO3bUunXr9N5778Wb18HBwbqOK1euqE+fPnrttdf02muvSbq/L73//vvWv7mvr69Gjx6td999VyNHjnxkbQCAzI8z3QCALOWll17S2bNntWTJErVs2VLBwcGqXr26goKCkpzvwVBeoEABSdLFixclSYcPH1bt2rVt+j/8+kE3b97U8ePH1bt3b+vZdxcXF40ZM0bHjx9P5Zbdt3fvXq1bt85muWXLlpUkm2XXqFEj3rxr1qxR06ZNVahQIbm6uurVV1/VlStXbIZOP8qhQ4dUpEgRa+CWpPLly8vDw0OHDh2ytvn4+FgDt3T/dxr3+0yMYRjJrqNo0aLWwC1JdevWVWxsrA4fPpzsZcR58G//1FNPKVeuXNbAHdf2qNrv3r2rl156ScWKFdNXX31lbd+7d69GjRpl8/eKG72Qkt87ACDz4kw3ACDLcXJyUrNmzdSsWTN99NFH6tOnj0aOHGlzne3DHrwbtcVikXT/bG1qxF0n/f3336tOnTo20+zt7VO1zAeX3bZtW33yySfxpsV9WSAp3t3GT548qeeff15vvPGGxo4dqzx58ujvv/9W7969defOHeXKleux6nrYw3f3tlgsSf4+S5cubRPaHxTXHncWP609/LdPae2S9MYbbygsLEzbt29Xjhz/9/Hqxo0bCgwMtI54eJCTk9NjVg4AyAwI3QCALK98+fKP9VzuMmXKaMeOHTZtD79+0FNPPaWCBQvqxIkTeuWVV1K9XgcHB8XExNi0Va9eXb/99pt8fHxswt2j/PPPP4qNjdXnn38uO7v7A93mzZv3yPU9rFy5cgoLC1NYWJj1bPfBgwd17do1lS9fPtn1PKxLly56+eWX9ccff8S7rvvzzz+Xl5eXmjVrZm07ffq0zp49q4IFC0qStm7dKjs7O5UpUybZ25JWJk6cqHnz5mnz5s3y8vKymVa9enUdPnxYJUuWfCK1AAAyHoaXAwCyjCtXrujZZ5/Vzz//rH///VehoaGaP3++Pv30U7344oupXu6bb76pZcuWaeLEiTp69KimTp2q5cuXW8+IJyQwMFDjxo3T119/rSNHjmjfvn2aOXOmJk6cmOz1+vj4aNu2bTp58qQuX76s2NhYDRw4UFevXlXXrl21Y8cOHT9+XCtXrlSvXr2SDJklS5bU3bt39c033+jEiRP66aefrDdYe3B9N27c0Nq1a3X58uUEhz8/99xzqlSpkl555RXt2rVL27dvV/fu3dWoUSPVrFkz2dv2sC5duqh9+/bq0aOHpk+frpMnT+rff/9Vv379tGTJEv3www82Z++dnJzUo0cP7d27Vxs3btRbb70lPz8/6/XVPj4++vfff3X48GFdvnxZd+/eTXVtSVmzZo3effddffbZZ8qbN6/Onz+v8+fPW++kPmLECP34448KDAzUgQMHdOjQIc2ZM0cffvihKfUAADIeQjcAIMtwcXFRnTp19MUXX6hhw4aqWLGiPvroI/Xt21eTJk1K9XLr16+vKVOmaOLEiapSpYpWrFihoUOHJjk8uE+fPvrhhx80c+ZMVapUSY0aNVJQUJCKFy+e7PUOGzZM9vb2Kl++vLy9vXX69GkVLFhQmzZtUkxMjJo3b65KlSppyJAh8vDwsJ7BTkiVKlU0ceJEffLJJ6pYsaJmz56tcePG2fSpV6+e+vfvr86dO8vb2zvejdik+0OtFy9eLE9PTzVs2FDPPfecfH19NXfu3GRvV0IsFovmzZunDz74QF988YXKlCmjBg0a6NSpUwoODo73jO6SJUuqQ4cOat26tZo3b67KlSvbPJKtb9++KlOmjGrWrClvb29t2rTpsepLzN9//62YmBj1799fBQoUsP4MHjxYktSiRQstXbpUq1atUq1atfT000/riy++ULFixUypBwCQ8ViMlNy1JIOLjIyUu7u7IiIi5Obmlt7lAECmFRUVpdDQUBUvXpzrThPRt29fhYSEaOPGjeldSrYTEBCg33//3fq8baQMxzcApI3k5k+u6QYAIBkmTJigZs2aKXfu3Fq+fLlmzZplc2YVAAAgIYRuAACSYfv27fr00091/fp1+fr66uuvv1afPn3SuywAAJDBMbwcABAPw0+BrIvjGwDSRnLzJzdSAwAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgDgESwWi37//fck+/Ts2VPt2rV7IvVkNSdPnpTFYtGePXvSu5THFhQUJA8Pj8deTnL2OQBA5pAjvQsAAGQub64d/ETX903Tr1LUv2fPnpo1a5b69eunKVOm2EwbOHCgJk+erB49eigoKChV9Zw8eVLFixfX7t27VbVqVWv7V199JcMwkpzXx8dHQ4YM0ZAhQ1K17rTwcA0Wi0WLFi16Yl8Y9OzZU9euXbMJlEWKFNG5c+eUN29e09d/9epVjRo1SosWLbKus2XLlgoICFDRokVTtKyE/p6dO3dW69atH7vOc+fOydPT87GXAwBIf5zpBgBkOUWKFNGcOXN0+/Zta1tUVJR++eWXFAer5HJ3d0+TM5zJcefOnSeynpS4e/duque1t7dX/vz5lSOHuecCrl69qqefflpr1qzRlClTdOzYMc2ZM0fHjh1TrVq1dOLEicdeh7Ozs/Lly/fYy8mfP78cHR0fezkAgPRH6AYAZDnVq1dXkSJFtHDhQmvbwoULVbRoUVWrVs2mr4+Pj7788kubtqpVqyogICDBZRcvXlySVK1aNVksFjVu3FjSo4eXN27cWKdOndLQoUNlsVhksVgkSQEBATZnzCXpyy+/lI+Pj/V13LLHjh2rggULqkyZMtYh2QsXLlSTJk2UK1cuValSRVu2bEn8F/OQuHW0b99eFovFZp2LFy9W9erV5eTkJF9fXwUGBurevXvW6RaLRd99951eeOEF5c6dW2PHjlVMTIx69+6t4sWLy9nZWWXKlNFXX/3fSIWAgADNmjVLixcvtv4OgoODExxevn79etWuXVuOjo4qUKCA3n//fZv1N27cWG+99Zbeffdd5cmTR/nz50/0bxZn+PDhOnv2rNasWaNWrVqpaNGiatiwoVauXKmcOXNq4MCBNssfNGiQBg0aJHd3d+XNm1cfffSRdTRDYn/Ph4eXx/19Z8yYoaJFi8rFxUUDBgxQTEyMPv30U+XPn1/58uXT2LFjbWp9cHh5QECAdR0P/sSN1oiNjdW4ceOsv/cqVapowYIFSf4uAABPDqEbAJAlvfbaa5o5c6b19YwZM9SrV6/HXu727dslSWvWrNG5c+dsgn1SFi5cqMKFC2vUqFE6d+6czp07l6L1rl27VocPH9bq1au1dOlSa/vw4cM1bNgw7dmzR6VLl1bXrl1twmlSduzYIUmaOXOmzp07Z329ceNGde/eXYMHD9bBgwc1depUBQUFxQuGAQEBat++vfbt26fXXntNsbGxKly4sObPn6+DBw9qxIgR+uCDDzRv3jxJ0rBhw+Tn56eWLVtafwf16tWLV9d///2n1q1bq1atWtq7d6++++47TZ8+XWPGjLHpN2vWLOXOnVvbtm3Tp59+qlGjRmn16tUJbmtsbKzmzJmjV155Rfnz57eZ5uzsrAEDBmjlypW6evWqzfJz5Mih7du366uvvtLEiRP1ww8/SErZ3/P48eNavny5VqxYoV9//VXTp09XmzZtdObMGa1fv16ffPKJPvzwQ23bti3B+YcNG2Zdx7lz5zRhwgTlypVLNWvWlCSNGzdOP/74o6ZMmaIDBw5o6NCh6tatm9avX59oTQCAJ4drugEAWVK3bt3k7++vU6dOSZI2bdqkOXPmKDg4+LGW6+3tLUny8vKKF96SkidPHtnb28vV1TVF88XJnTu3fvjhBzk4OEi6f225dD+QtWnTRpIUGBioChUq6NixYypbtuwjlxm3LR4eHjY1BQYG6v3331ePHj0kSb6+vho9erTeffddjRw50trv5ZdfjvdFRmBgoPX/ixcvri1btmjevHny8/OTi4uLnJ2dFR0dneTvYPLkySpSpIgmTZoki8WismXL6uzZs3rvvfc0YsQI2dndP2dQuXJlaz2lSpXSpEmTtHbtWjVr1izeMi9duqRr166pXLlyCa6zXLlyMgxDx44dU+3atSXdv0zhiy++kMViUZkyZbRv3z598cUX6tu3b4r+nrGxsZoxY4ZcXV1Vvnx5NWnSRIcPH9ayZctkZ2enMmXK6JNPPtG6detUp06dePO7uLjIxcVFkrR161Z9+OGHmjVrlipWrKjo6Gh9/PHHWrNmjerWrSvp/t/r77//1tSpU9WoUaMkawMAmI/QDQDIkry9vdWmTRsFBQXJMAy1adPmidyoS5Jmz56tfv36WV8vX75cDRo0eKxlVqpUyRq4H1S5cmXr/xcoUECSdPHixWSF7sTs3btXmzZtsjmzHRMTo6ioKN26dUu5cuWSJOuZ1gd9++23mjFjhk6fPq3bt2/rzp078YbPP8qhQ4dUt25d65BtSapfv75u3LihM2fOWK/Lf3Dbpfvbf/HixSSX/aib3T3o6aeftqmhbt26+vzzzxUTEyN7e/tkL8fHx0eurq7W10899ZTs7e2tXx7EtT2q9tOnT6tdu3bWEQOSdOzYMd26dSveFw137tyJdykFACB9ELoBAFnWa6+9pkGDBkm6HwYTYmdnFy+IPc5NwSTphRdesDljWahQoUT7Jnf9uXPnTnD+nDlzWv8/LiDGxsamqN6H3bhxQ4GBgerQoUO8aU5OTonWNGfOHA0bNkyff/656tatK1dXV3322WeJDpt+XA9uu3R/+xPbdm9vb3l4eOjQoUMJTj906JAsFotKliz5ROpMSe2SdPPmTb3wwguqW7euRo0aZW2/ceOGJOnPP/+Mt59xIzYAyBgI3QCALKtly5a6c+eOLBaLWrRokWAfb29vm+txIyMjFRoamugy4842x8TEJNrH1dXV5szmg/M+PJ+3t7fOnz8vwzCsoflJPq86Z86c8WqqXr26Dh8+nOIAumnTJtWrV08DBgywth0/ftymT0K/g4eVK1dOv/32m83vZNOmTXJ1dVXhwoVTVFMcOzs7+fn5afbs2Ro1apTNkPDbt29r8uTJatGihfLkyWNtf/jLgq1bt6pUqVLWs9zJ2Za0YBiGunXrptjYWP300082Z9/Lly8vR0dHnT59mqHkAJBBcSM1AECWZW9vr0OHDungwYOJDgd+9tln9dNPP2njxo3at2+fevTokeTQ4Xz58snZ2VkrVqzQhQsXFBERkex6fHx8tGHDBv3333+6fPmypPt3wb506ZI+/fRTHT9+XN9++62WL1+esg19DD4+Plq7dq3Onz+v8PBwSdKIESP0448/KjAwUAcOHNChQ4c0Z84cffjhh0kuq1SpUtq5c6dWrlypI0eO6KOPPrLenO3B9f377786fPiwLl++nOBZ/QEDBigsLExvvvmmQkJCtHjxYo0cOVJvv/22zZDslPr444+VP39+NWvWTMuXL1dYWJg2bNigFi1a6O7du/FGQ5w+fVpvv/22Dh8+rF9//VXffPONBg/+v+fUJ/T3NENAQIDWrFmjqVOn6saNGzp//rzOnz+v27dvy9XVVcOGDdPQoUM1a9YsHT9+XLt27dI333yjWbNmmVYTACD5CN0AgCzNzc1Nbm5uiU739/dXo0aN9Pzzz6tNmzZq166dSpQokWj/HDly6Ouvv9bUqVNVsGBBvfjii8muZdSoUTp58qRKlChhvYlZuXLlNHnyZH377beqUqWKtm/frmHDhiV/Ax/T559/rtWrV6tIkSLWa4BbtGihpUuXatWqVapVq5aefvppffHFFypWrFiSy+rXr586dOigzp07q06dOrpy5YrNWW9J6tu3r8qUKaOaNWvK29tbmzZtirecQoUKadmyZdq+fbuqVKmi/v37q3fv3o8M/Y/i5eWlrVu3qkmTJurXr59KlCghPz8/lShRQjt27JCvr69N/+7du+v27duqXbu2Bg4cqMGDB+v111+3Tk/o72mG9evX68aNG6pXr54KFChg/Zk7d64kafTo0froo480btw4lStXTi1bttSff/5pfbwdACB9WYyU3FEkg4uMjJS7u7siIiKS/IAFAEhaVFSUQkNDVbx4cZtreIHsonHjxqpatWq8Z7hnBRzfAJA2kps/OdMNAAAAAIBJCN0AAAAAAJiEu5cDAAA8JDg4OL1LAABkEZzpBgAAAADAJIRu4AkJCgqSxWKJ9/P+++8nOZ9hGBo/fryKFi0qZ2dn1a1bV1u3bn1CVSO7y0L32gTw/2W245r3T2QG2XU/za7bnVIMLweesBUrVsjd3d36ulChQkn2/+STTzRy5EiNHz9elStX1rfffqvmzZtrz5498R5vA6SVnDlzSpJu3bolZ2fndK4GQFq6deuWpP87zjML3j+RGWTX/TS7bndyEbqBJ6xGjRrKmzdvsvpGRUVp3LhxeueddzR06FBJUoMGDVS6dGlNmDBBkydPNrNUZGP29vby8PDQxYsXJUm5cuWSxWJJ56oAPA7DMHTr1i1dvHhRHh4esre3T++SUoT3T2QG2XU/za7bnVyEbiAD27x5syIjI+Xn52dtc3BwUIcOHbRw4cJ0rAzZQf78+SXJGrwBZA0eHh7W4zur4v0TmUF23U+z43YTuoEnrEKFCrp8+bKKFSumvn376t133030bENISIgkqWzZsjbt5cqV0+nTp3X79m2G/sI0FotFBQoUUL58+XT37t30LgdAGsiZM2emO8Mdh/dPZAbZdT/NrtudXIRu4AkpUKCAAgMDVadOHVksFi1ZskQffvih/vvvP02aNCnBecLDw+Xo6CgnJyebdk9PTxmGofDw8Cz3jxIyHnt7+0z7IR1A5sf7JzKD7LqfZtftTilCN/CEtGjRQi1atLC+bt68uZydnfXFF19o+PDhKlCgQDpWBwBAxsT7JzKD7LqfZtftTikeGQakIz8/P8XExGjPnj0JTvf09FR0dLSioqJs2sPDw2WxWOTp6fkEqgQAIGPh/ROZQXbdT7PrdieF0A1kYHHXuhw+fNimPSQkxPpcQwAAYIv3T2QG2XU/zY7bTegG0tGcOXNkb2+vatWqJTi9Xr16cnNz0/z5861td+/e1cKFC9W6desnVSYAABkK75/IDLLrfppdtzspGeqa7oCAAAUGBtq0lSlTxnqHOyAza9GihZ599llVqlRJkrRkyRJNmzZNgwcPtj66pWnTpjp16pSOHTsmSXJycpK/v78CAgLk7e2tSpUqafLkybpy5YqGDRuWbtsCAMCTwvsnMoPsup9m1+1OqQwVuqX7t5tfs2aN9XWOHBmuRCBVypYtq+nTp+vMmTOKjY1V6dKl9eWXX+rNN9+09omJidG9e/ds5nvvvfdkGIYmTJigS5cuqWrVqlq5cqV8fX2f9CYAAPDE8f6JzCC77qfZdbtTymIYhpHeRcQJCAjQ77//nuhF948SGRkpd3d3RUREyM3NLW2LAwAAAADg/0tu/sxw13QfPXpUBQsWlK+vr1555RWdPn06vUsCAAAAACBVMtTY7Tp16igoKEhlypTRuXPnFBgYqAYNGmj//v1ydXWN1z86OlrR0dHW15GRkU+yXAAAAAAAkpShhpc/7Nq1aypWrJgmTpyo3r17x5ue0I3XJDG8HGnizbWD07uEdPFN06/SuwQAQCbG+ycyg6pjAtK7hHTRoG54epeQLsw6PjPt8PIHeXh4qHTp0tY73T3M399fERER1p+wsLAnXCEAAAAAAInL0KH7xo0bOn78uAoUKJDgdEdHR7m5udn8AAAAAACQUWSo0D1s2DCtX79eJ0+e1ObNm9W+fXvZ29ura9eu6V0aAAAAAAAplqFupHbmzBl17dpVV65ckbe3t5555hlt3bpV3t7e6V0aAAAAAAAplqFC95w5c9K7BAAAAAAA0kyGGl4OAAAAAEBWQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgBAom7cuKHChQvLYrFo586dSfY1DEPjx49X0aJF5ezsrLp162rr1q1PqNK0lV23G5kL+ykyA/ZTgNANAEjC6NGjde/evWT1/eSTTzRy5EgNHTpUS5cuVYECBdS8eXOdOHHC5CrTXnbdbmQu7KfIDNhPAUI3ACARISEh+vbbbxUYGPjIvlFRURo3bpzeeecdDR06VE2bNtWcOXOUJ08eTZgw4QlUm3ay63Yjc2E/RWbAfgrcR+gGACTozTffVP/+/VWmTJlH9t28ebMiIyPl5+dnbXNwcFCHDh20bNkyM8tMc9l1u5G5sJ8iM2A/Be4jdAMA4lmwYIH27dunESNGJKt/SEiIJKls2bI27eXKldPp06d1+/btNK/RDNl1u5G5sJ8iM2A/Bf4PoRsAYOPWrVt6++239fHHH8vNzS1Z84SHh8vR0VFOTk427Z6enjIMQ+Hh4WaUmqay63Yjc2E/RWbAfgrYInQDAGyMGTNGTz31lHr16pXepTxR2XW7kbmwnyIzYD8FbOVI7wIAABnHqVOn9Pnnn2vRokWKiIiQdP9xL3H/vXHjhlxcXOLN5+npqejoaEVFRdmcpQgPD5fFYpGnp+eT2YBUyq7bjcyF/RSZAfspEB+hGwBgFRoaqjt37qhNmzbxpjVp0kR16tRJ8JmpcdfgHT58WFWqVLG2h4SEWJ+3mpFl1+1G5sJ+isyA/RSIj9ANALCqWrWq1q1bZ9O2Z88eDR06VFOmTFGtWrUSnK9evXpyc3PT/PnzrR+W7t69q4ULF6p169am1/24sut2I3NhP0VmwH4KxJdhQ/f48ePl7++vwYMH68svv0zvcgAgW/Dw8FDjxo0TnFajRg1Vr15dktS0aVOdOnVKx44dkyQ5OTnJ399fAQEB8vb2VqVKlTR58mRduXJFw4YNe1Llp1p23W5kLuynyAzYT4H4MmTo3rFjh6ZOnarKlSundykAgATExMTo3r17Nm3vvfeeDMPQhAkTdOnSJVWtWlUrV66Ur69vOlWZ9rLrdiNzYT9FZsB+iuzEYhiGkd5FPOjGjRuqXr26Jk+erDFjxqhq1arJPtMdGRkpd3d3RUREJPvxBEBi3lw7OL1LSBffNP0qvUsAAGRivH8iM6g6JiC9S0gXDepmz0evmXV8Jjd/ZrhHhg0cOFBt2rTRc889l96lAAAAAADwWDLU8PI5c+Zo165d2rFjR7L6R0dHKzo62vo6MjLSrNIAAAAAAEixDBO6w8LCNHjwYK1evdrm2XxJGTdunAIDA02uDAAyLobHZS8MX81csu/xmd4VICWadx6V3iWkj0rpXQCykwwzvPyff/7RxYsXVb16deXIkUM5cuTQ+vXr9fXXXytHjhyKiYmJN4+/v78iIiKsP2FhYelQOQAAAAAACcswZ7qbNm2qffv22bT16tVLZcuW1XvvvSd7e/t48zg6OsrR0fFJlQgAAAAAQIpkmNDt6uqqihUr2rTlzp1bXl5e8doBAAAAAMgMMszwcgAAAAAAspoMc6Y7IcHBweldAgAAAAAAqcaZbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAEyS43Fmvnz5si5fviyLxaK8efPKy8srreoCAAAAACDTS1HovnnzpubPn6/Fixdr8+bNunz5ss30vHnzqm7dumrXrp06deqk3Llzp2mxAAAAAABkJskK3VeuXNG4ceM0depURUVFqXLlynrxxRfl6+srT09PGYah8PBwhYaG6p9//lHfvn315ptvql+/fnr//feVN29es7cDAAAAAIAMJ1mh28fHRyVLltRnn32ml156Sd7e3kn2v3Tpkn777TdNmzZN06ZNU2RkZJoUCwAAAABAZpKs0L1gwQK1aNEi2Qv19vZW//791b9/f61cuTLVxQEAAAAAkJkl6+7lKQncaTkvAAAAAACZGY8MAwAAAADAJI/1yLDY2FgtWLBAa9as0dWrV+Xt7a1WrVrphRdeSKv6AAAAAADItJIdusuXL6/PPvtMbdq0kXT/8WGtWrXSpk2bZLFY5OXlpcuXL2vatGlq1aqVFi9eLHt7e9MKBwAAAAAgo0v28PKQkBBFRERYX7/33nv6+++/NWbMGN24cUMXLlxQRESE3nnnHS1btkyff/65KQUDAAAAAJBZpPqa7l9//VU9e/aUv7+/nJycJEkuLi769NNP1apVK/38889pViQAAAAAAJlRqkL39evXFR4erpYtWyY4vWXLljp27NhjFQYAAAAAQGaXotBtsVgkSblz51auXLlkZ5f47FzPDQAAAADI7lIUunv37i03Nzd5eHgoKipKu3btSrBfSEiIChYsmCYFAgAAAACQWSX77uU9evSI1xZ35vtBN27c0K+//qq2bds+XmUAAAAAAGRyyQ7dM2fOTFa/nDlzavfu3fLw8EhtTQAAAAAAZAnJDt3J5ejoqGLFiqX1YgEAAAAAyHRS/ciwB12+fFm+vr7asmVLWiwOAAAAAIAsIU1Cd0xMjE6ePKnbt2+nxeIAAAAAAMgS0iR0AwAAAACA+AjdAAAAAACYJE1Ct4uLi0aOHClfX9+0WBwAAAAAAFlCmty9PHfu3Bo5cmRaLAoAAAAAgCzDlOHlYWFh2rBhgxmLBgAAAAAg0zAldP/4449q0qSJGYsGAAAAACDT4EZqAAAAAACYJNnXdI8aNSrZC12/fn2qigEAAAAAICtJdugOCAiQxWKRYRjJ6m+xWFJdFAAAAAAAWUGyQ3e+fPlUvXp1/fTTT4/s+/nnn+uTTz55rMIAAAAAAMjskh2669Spo507d8rLy+uRfXPnzv1YRQEAAAAAkBUk+0ZqtWvX1rlz53T69OlH9i1WrJgaNmz4WIUh61q2bJkaNWokb29vOTo6ytfXV2+//bYiIiIeOe/06dNVunRpOTk5qUqVKlq6dOkTqBjZEfspAAAA0kKyQ/fw4cMVGxurokWLPrJvt27dtG7duscqDFnX1atXVadOHU2ZMkUrV67U22+/rR9//FGdOnVKcr45c+aob9++6ty5s5YvX666deuqffv22rp16xOqHNkJ+ykAAADSQrKHlwNppVu3bjavGzduLEdHR73++us6e/asChYsmOB8I0eOVJcuXTR69GhJUpMmTfTvv/9q1KhRWrZsmel1I3thPwUAAEBa4DndyBDi7hVw586dBKefOHFCR44ckZ+fn017ly5dtHbtWkVHR5teI8B+CgAAgJRKVuju16+fQkNDU7zw48ePq1+/fimeD9lDTEyMoqKitGvXLo0aNUovvPCCfHx8EuwbEhIiSSpbtqxNe7ly5XTnzp1U7Z9AcrCfAgAA4HEkK3SHhYWpTJkyatWqlYKCghQWFpZo35MnT+qHH35Q8+bNVbZsWZ05cybNikXWUqxYMTk7O6tGjRoqUKCAfvnll0T7hoeHS5I8PDxs2j09PSXdv/4WMAP7KQAAAB5Hsq7pXrZsmTZt2qQJEybo9ddfV0xMjLy8vOTj4yNPT08ZhqHw8HCFhoYqPDxc9vb2at26tdatW6dnnnnG7G1AJrVs2TLdvHlTBw4c0JgxY9S2bVutXr1a9vb26V0aYMV+CgAAgMeR7Bup1a9fX/Xr19elS5e0dOlSbdmyRSEhIdYz2V5eXurQoYPq1q2rNm3aKF++fKYVjayhcuXKkqS6deuqVq1aqlq1qhYtWqSOHTvG6xt3pjAiIkL58+e3tsedWcyTJ88TqBjZEfspAAAAHkeK717u7e2tXr16qVevXmbUg2yqcuXKypkzp44dO5bg9LhrZENCQlSmTBlre0hIiBwcHOTr6/tE6kT2xn4KAACAlOLu5cgQtm3bprt37yYaSnx9fVW6dGnNnz/fpn3u3Llq2rSpHBwcnkSZyObYTwEAAJBSGeo53d99952+++47nTx5UpJUoUIFjRgxQq1atUrfwpCmOnTooJo1a6py5cpydnbW3r179dlnn6ly5cpq166dJKl3796aNWuW7t27Z50vICBAr7zyikqUKKEmTZpo7ty52rZtmzZs2JBOW4KsjP0UAAAAaSFDhe7ChQtr/PjxKlWqlAzD0KxZs/Tiiy9q9+7dqlChQnqXhzRSu3ZtzZ07V+PHj1dsbKx8fHzUt29fDRs2zHomMCYmRjExMTbzde3aVbdu3dL48eM1fvx4lSlTRosWLVLdunXTYzOQxbGfAgAAIC1YDMMw0ruIpOTJk0efffaZevfu/ci+kZGRcnd3V0REhNzc3J5AdcjK3lw7OL1LSBffNP0qvUtAClQdE5DeJaSLBnXD07uEdMHxmblwfGYvmfX4bN55VHqXkC4uVopN7xLSBcdn2kpu/sxQZ7ofFBMTo/nz5+vmzZuJniGKjo5WdHS09XVkZOSTKg8AAAAAgEfKcKF73759qlu3rqKiouTi4qJFixapfPnyCfYdN26cAgMDn3CF2U/2/aY+vStASmTXb+pVKb0LAAAAQFIe6+7lW7du1bhx4zR06FAdPXpUknTr1i3t2rVLN27cSNUyy5Qpoz179mjbtm1644031KNHDx08eDDBvv7+/oqIiLD+hIWFpXpbAAAAAABIa6k6033nzh116dJFixcvlmEYslgsatu2rUqVKiU7Ozs1b95cQ4cO1fDhw1O8bAcHB5UsWVKSVKNGDe3YsUNfffWVpk6dGq+vo6OjHB0dU7MJAAAAAACYLlVnuj/66CMtXbpU3333nQ4fPqwH78Xm5OSkTp06afHixWlSYGxsrM112wAAAAAAZBapOtP966+/6o033tDrr7+uK1euxJterlw5zZ8/P8XL9ff3V6tWrVS0aFFdv35dv/zyi4KDg7Vy5crUlAkAAAAAQLpKVei+ePGiKlVK/O499vb2unXrVqqW2717d507d07u7u6qXLmyVq5cqWbNmqWmTAAAAAAA0lWqQneRIkUUEhKS6PRNmzZZr8tOienTp6emHAAAAAAAMqRUXdP98ssva+rUqdqyZYu1zWKxSJK+//57zZs3T927d0+bCgEAAAAAyKRSdaZ7+PDh2rp1qxo2bKhy5crJYrFo6NChunr1qs6cOaPWrVtr6NChaV0rAAAAAACZSqrOdDs4OGjFihWaOXOmfH19VbZsWUVHR6ty5coKCgrSH3/8IXt7+7SuFQAAAACATCXFZ7pv376t4cOHq0mTJurWrZu6detmRl0AAAAAAGR6KT7T7ezsrKlTp+rChQtm1AMAAAAAQJaRquHlNWrU0P79+9O6FgAAAAAAspRUhe4vv/xSc+bM0Q8//KB79+6ldU0AAAAAAGQJqbp7ec+ePWVnZ6d+/frprbfeUqFCheTs7GzTx2KxaO/evWlSJAAAAAAAmVGqQneePHnk5eWlMmXKpHU9AAAAAABkGakK3cHBwWlcBgAAAAAAWU+qrukGAAAAAACPlqoz3ZIUExOjn3/+WX/++adOnTolSSpWrJief/55vfLKK7K3t0+zIgEAAAAAyIxSdaY7IiJC9evX12uvvaZVq1bp7t27unv3rlavXq1evXrpmWeeUWRkZFrXCgAAAABAppKq0D18+HD9888/+uabb3Tp0iXt2rVLu3bt0sWLFzVp0iTt3LlTw4cPT+taAQAAAADIVFIVuhctWqQBAwZowIABypkzp7U9Z86ceuONN/TGG2/ot99+S7MiAQAAAADIjFIVuq9cuZLk48LKli2rq1evprooAAAAAACyglSF7pIlS2rJkiWJTl+yZIlKlCiR6qIAAAAAAMgKUhW6BwwYoFWrVql169ZatWqVTp48qZMnT2rlypVq06aNVq9erUGDBqV1rQAAAAAAZCqpemTYgAEDdPHiRY0fP14rV660mZYzZ06NGDFCb7zxRpoUCAAAAABAZpXq53QHBARo0KBBWrNmjc1zup977jnlzZs3zQoEAAAAACCzSnXolqS8efOqS5cuaVULAAAAAABZSqqu6V6zZo0++OCDRKcPHz5cf/31V6qLAgAAAAAgK0hV6B49erTCwsISnf7ff/9pzJgxqS4KAAAAAICsIFWhe9++fapTp06i02vVqqV///031UUBAAAAAJAVpCp0R0dH686dO0lOv3XrVqqLAgAAAAAgK0hV6K5YsaIWLVqU4DTDMLRw4UKVL1/+sQoDAAAAACCzS1XofvPNN7Vp0yZ16tRJ+/bt071793Tv3j39+++/6tSpk7Zs2aI333wzrWsFAAAAACBTSdUjw7p166bjx49r9OjRWrhwoezs7mf32NhYWSwWffjhh+rRo0eaFgoAAAAAQGaT6ud0jxw5Ut26ddOiRYt04sQJSVKJEiXUrl07lShRIs0KBAAAAAAgs0p16Jbuh+xhw4alVS0AAAAAAGQpjxW644SEhGj+/Pk6d+6cypYtq549e8rNzS0tFg0AAAAAQKaV7NA9adIkff3119q8ebPy5s1rbf/jjz/UqVMnm0eIff3119q6datNPwAAAAAAsptk3718yZIlKlGihE2Qvnfvnvr06SN7e3vNnDlT+/bt0/jx43Xq1CmNHTvWlIIBAAAAAMgskh26Dx48qKefftqmbd26dbp06ZKGDh2qHj16qEKFCnr33Xfl5+enZcuWpXmxAAAAAABkJskO3VeuXFGRIkVs2tauXSuLxaL27dvbtNevX1+nT59OmwoBAAAAAMikkh26n3rqKZ0/f96mbePGjcqVK5eqVKli0+7g4CAHB4e0qRAAAAAAgEwq2aG7Zs2amjVrlq5fvy5JOnDggLZv364WLVooRw7b+7GFhISocOHCaVspAAAAAACZTLLvXj5y5EjVqlVLpUqVUoUKFfTPP//IYrHI398/Xt9Fixbp2WefTdNCAQAAAADIbJJ9prtSpUr666+/VKNGDZ09e1ZPP/20li1bpho1atj0Cw4OVq5cudSpU6c0LxYAAAAAgMwk2We6JalevXr6888/k+zTuHFj7du377GKAgAAAAAgK0j2mW4AAAAAAJAyhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQuhOR/Pnz9eLL76owoULK3fu3KpatapmzJghwzCSnM8wDI0fP15FixaVs7Oz6tatq61btz6hqgEAAAAAyUXoTkcTJ05Urly59Pnnn+uPP/5Qq1at1LdvX40aNSrJ+T755BONHDlSQ4cO1dKlS1WgQAE1b95cJ06ceEKVAwAAAACSI0d6F5Cd/fHHH8qbN6/19bPPPqsrV65o4sSJ+uijj2RnF/87kaioKI0bN07vvPOOhg4dKklq0KCBSpcurQkTJmjy5MlPrH4AAAAAQNI4052OHgzccapVq6bIyEjdvHkzwXk2b96syMhI+fn5WdscHBzUoUMHLVu2zLRaAQAAAAApR+jOYP7++28VKlRIrq6uCU4PCQmRJJUtW9amvVy5cjp9+rRu375teo0AAAAAgOQhdGcgf//9t+bMmaNhw4Yl2ic8PFyOjo5ycnKyaff09JRhGAoPDze7TAAAAABAMhG6M4gzZ86oc+fOatKkid566630LgcAAAAAkAYI3RnAtWvX1KpVK3l5eem3335L8AZqcTw9PRUdHa2oqCib9vDwcFksFnl6eppdLgAAAAAgmQjd6ez27dt6/vnnFRERoeXLl8vd3T3J/nHXch8+fNimPSQkxPrcbgAAAABAxkDoTkf37t2Tn5+fDh06pBUrVqhQoUKPnKdevXpyc3PT/PnzrW13797VwoUL1bp1azPLBQAAAACkUIYK3ePGjVOtWrXk6uqqfPnyqV27dvHO6GYlAwYM0NKlSzV8+HBFRkZq69at1p/o6GhJUtOmTVWyZEnrPE5OTvL399eECRP01Vdf6a+//lLXrl115cqVJG/ABgAAAAB48nKkdwEPWr9+vQYOHKhatWrp3r17+uCDD9S8eXMdPHhQuXPnTu/y0tyqVaskSe+88068aaGhofLx8VFMTIzu3btnM+29996TYRiaMGGCLl26pKpVq2rlypXy9fV9InUDAAAAAJInQ4XuFStW2LwOCgpSvnz59M8//6hhw4bpVJV5Tp48+cg+wcHB8dosFov8/f3l7++f9kUBAAAAANJMhgrdD4uIiJAk5cmTJ8Hp0dHR1mHYkhQZGflE6gIAAAAAIDkybOiOjY3VkCFDVL9+fVWsWDHBPuPGjVNgYOATq6l551FPbF0ZSqX0LgAAAAAAMqcMdSO1Bw0cOFD79+/XnDlzEu3j7++viIgI609YWNgTrBAAAAAAgKRlyDPdgwYN0tKlS7VhwwYVLlw40X6Ojo5ydHR8gpUBAAAAAJB8GSp0G4ahN998U4sWLVJwcLCKFy+e3iUBAAAAAJBqGSp0Dxw4UL/88osWL14sV1dXnT9/XpLk7u4uZ2fndK4OAAAAAICUyVDXdH/33XeKiIhQ48aNVaBAAevP3Llz07s0AAAAAABSLEOd6TYMI71LAAAAAAAgzWSoM90AAAAAAGQlhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAmc6xY8fUv39/Va1aVTly5FDFihWTNZ9hGBo/fryKFi0qZ2dn1a1bV1u3bjW5WgBAdkboBgAAmc6BAwf0559/qmTJkipfvnyy5/vkk080cuRIDR06VEuXLlWBAgXUvHlznThxwsRqAQDZGaEbAABkOm3btlVYWJgWLFig6tWrJ2ueqKgojRs3Tu+8846GDh2qpk2bas6cOcqTJ48mTJhgcsUAgOyK0A0AADIdO7uUf4TZvHmzIiMj5efnZ21zcHBQhw4dtGzZsrQsDwAAK0I3AADIFkJCQiRJZcuWtWkvV66cTp8+rdu3b6dHWQCALI7QDQAAsoXw8HA5OjrKycnJpt3T01OGYSg8PDydKgMAZGWEbgAAAAAATELoBgAA2YKnp6eio6MVFRVl0x4eHi6LxSJPT890qgwAkJURugEAQLYQdy334cOHbdpDQkKsz+0GACCtEboBAEC2UK9ePbm5uWn+/PnWtrt372rhwoVq3bp1OlYGAMjKMlTo3rBhg9q2bauCBQvKYrHo999/T++SAABABnTr1i0tWLBACxYs0KlTpxQZGWl9fenSJUlS06ZNVbJkSes8Tk5O8vf314QJE/TVV1/pr7/+UteuXXXlyhUNGzYsvTYFAJDF5UjvAh508+ZNValSRa+99po6dOiQ3uUAAIAM6uLFi+rUqZNNW9zrdevWqXHjxoqJidG9e/ds+rz33nsyDEMTJkzQpUuXVLVqVa1cuVK+vr5PrHYAQPaSoUJ3q1at1KpVq/QuAwAAZHA+Pj4yDCPJPsHBwfHaLBaL/P395e/vb1JlAADYylDDywEAAAAAyEoy1JnulIqOjlZ0dLT1dWRkZDpWAwAAAACArUwduseNG6fAwMD0LgMAACSheedR6V1C+qiU3gUAADKCTD283N/fXxEREdafsLCw9C4JAAAAAACrTH2m29HRUY6OjuldBgAAAAAACcpQofvGjRs6duyY9XVoaKj27NmjPHnyqGjRoulYGQAAAAAAKZehQvfOnTvVpEkT6+u3335bktSjRw8FBQWlU1UAAAAAAKROhgrdjRs3fuQzNwEAAAAAyCwy9Y3UAAAAAADIyAjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJMmTo/vbbb+Xj4yMnJyfVqVNH27dvT++SAAAAAABIsQwXuufOnau3335bI0eO1K5du1SlShW1aNFCFy9eTO/SAAAAAABIkQwXuidOnKi+ffuqV69eKl++vKZMmaJcuXJpxowZ6V0aAAAAAAApkiO9C3jQnTt39M8//8jf39/aZmdnp+eee05btmyJ1z86OlrR0dHW1xEREZKkyMhIU+q7dzfKlOVmdDFRRnqXkC7u3Ix+dKcsyKzjx2wcn9kLx2fmwvGZvXB8Zi4cn9kLx6c5yzWMpPcni/GoHk/Q2bNnVahQIW3evFl169a1tr/77rtav369tm3bZtM/ICBAgYGBT7pMAAAAAAAkSWFhYSpcuHCi0zPUme6U8vf319tvv219HRsbq6tXr8rLy0sWiyUdK0NaiIyMVJEiRRQWFiY3N7f0LgfAAzg+gYyL4xPIuDg+sxbDMHT9+nUVLFgwyX4ZKnTnzZtX9vb2unDhgk37hQsXlD9//nj9HR0d5ejoaNPm4eFhZolIB25ubvyjBGRQHJ9AxsXxCWRcHJ9Zh7u7+yP7ZKgbqTk4OKhGjRpau3attS02NlZr1661GW4OAAAAAEBmkKHOdEvS22+/rR49eqhmzZqqXbu2vvzyS928eVO9evVK79IAAAAAAEiRDBe6O3furEuXLmnEiBE6f/68qlatqhUrVuipp55K79LwhDk6OmrkyJHxLiEAkP44PoGMi+MTyLg4PrOnDHX3cgAAAAAAspIMdU03AAAAAABZCaEbAAAAAACTELoBAAAAADAJoTsbCggIkMViSe8yEhUUFCSLxaKTJ0+mdykAnrDGjRurcePG6V0GYJVR3jMzSh3pyWKxKCAgIL3LABLE51ckhdANJGLz5s0KCAjQtWvX0rsUQJL0yy+/6Msvv8z2NQAAkJUsW7aML5SyOEI3MpxXX31Vt2/fVrFixdK1js2bNyswMJDQjQwjIwRes2tYtWqVVq1aZdrygZT68MMPdfv27fQuA0AWtmzZMgUGBqZ3GTARoRsZjr29vZycnLL9MDrgcURFRSk2Nja9y0i2W7duSZIcHBzk4OCQztUA/ydHjhxycnJK7zIAAJkYoTsD+vfff2WxWLRkyRJr2z///COLxaLq1avb9G3VqpXq1Kljfb18+XI1aNBAuXPnlqurq9q0aaMDBw4ka70///yzateurVy5csnT01MNGzaMd8Zp8uTJqlChghwdHVWwYEENHDgw3pngxo0bq2LFivr333/VqFEj5cqVSyVLltSCBQskSevXr1edOnXk7OysMmXKaM2aNTbzJ3RNjI+Pj55//nn9/fffql27tpycnOTr66sff/wxwd9fo0aN5OzsrMKFC2vMmDGaOXNmiq6zCQgI0P/+9z9JUvHixWWxWGzmv3fvnkaPHq0SJUrI0dFRPj4++uCDDxQdHW2znJTUfe3aNQ0ZMkRFihSRo6OjSpYsqU8++SRTBSc8nuvXr2vIkCHy8fGRo6Oj8uXLp2bNmmnXrl1q3Lix/vzzT506dcq6P/r4+EiSgoODZbFYNGfOHH344YcqVKiQcuXKpcjIyESvA03s2rPly5erUaNGcnV1lZubm2rVqqVffvlFkpKsIbHlxdUWHBxsbYv7N+Kff/5Rw4YNlStXLn3wwQfWaQ9e0x03/7x58zR27FgVLlxYTk5Oatq0qY4dOxZvu7799lv5+vrK2dlZtWvX1saNG7lOHAlasGCBLBaL1q9fH2/a1KlTZbFYtH///gSPodWrV+uZZ56Rh4eHXFxcVKZMGes+LKXseNi4caM6deqkokWLytHRUUWKFNHQoUPT5Oz6v//+q549e8rX11dOTk7Knz+/XnvtNV25csWmX9w2hoSEyM/PT25ubvLy8tLgwYMVFRVl09disWjQoEGaPXu2ypQpIycnJ9WoUUMbNmyIt/7//vtPr732mp566ik5OjqqQoUKmjFjhk2fO3fuaMSIEapRo4bc3d2VO3duNWjQQOvWrXvs7Ufm9t9//6l3794qWLCgHB0dVbx4cb3xxhu6c+eOJOnEiRPq1KmT8uTJo1y5cunpp5/Wn3/+abOMB99DAgMDVahQIbm6uqpjx46KiIhQdHS0hgwZonz58snFxUW9evWK91kuJft8Qh712bxnz5769ttvreuK+4kTGxurL7/8UhUqVJCTk5Oeeuop9evXT+Hh4TbrMeMz55w5c1SjRg3rZ4JKlSrpq6++sk6/e/euAgMDVapUKTk5OcnLy0vPPPOMVq9enazfTXaSI70LQHwVK1aUh4eHNmzYoBdeeEHS/TdlOzs77d27V5GRkXJzc1NsbKw2b96s119/XZL0008/qUePHmrRooU++eQT3bp1S999952eeeYZ7d692/rhOCGBgYEKCAhQvXr1NGrUKDk4OGjbtm3666+/1Lx5c0n335QDAwP13HPP6Y033tDhw4f13XffaceOHdq0aZNy5sxpXV54eLief/55denSRZ06ddJ3332nLl26aPbs2RoyZIj69++vl19+WZ999pk6duyosLAwubq6Jvl7OXbsmDp27KjevXurR48emjFjhnr27KkaNWqoQoUKku7/A92kSRNZLBb5+/srd+7c+uGHH+To6Jiiv0GHDh105MgR/frrr/riiy+UN29eSZK3t7ckqU+fPpo1a5Y6duyod955R9u2bdO4ceN06NAhLVq0KMV137p1S40aNdJ///2nfv36qWjRotq8ebP8/f117ty5dB9SjCejf//+WrBggQYNGqTy5cvrypUr+vvvv3Xo0CENHz5cEREROnPmjL744gtJkouLi838o0ePloODg4YNG6bo6OgUnzEOCgrSa6+9pgoVKsjf318eHh7avXu3VqxYoZdffjlZNSTXlStX1KpVK3Xp0kXdunXTU089lWT/8ePHy87OTsOGDVNERIQ+/fRTvfLKK9q2bZu1z3fffadBgwapQYMGGjp0qE6ePKl27drJ09NThQsXTlWdyLratGkjFxcXzZs3T40aNbKZNnfuXFWoUEEVK1a0fmEc58CBA3r++edVuXJljRo1So6Ojjp27Jg2bdqUqjrmz5+vW7du6Y033pCXl5e2b9+ub775RmfOnNH8+fNTvX3S/S8HTpw4oV69eil//vw6cOCApk2bpgMHDmjr1q3xvkzw8/OTj4+Pxo0bp61bt+rrr79WeHh4vA/t69ev19y5c/XWW2/J0dFRkydPVsuWLbV9+3ZVrFhRknThwgU9/fTT1sDi7e2t5cuXq3fv3oqMjNSQIUMkSZGRkfrhhx/UtWtX9e3bV9evX9f06dPVokULbd++XVWrVn2s3wEyp7Nnz6p27dq6du2aXn/9dZUtW1b//fefFixYoFu3bik8PFz16tXTrVu39NZbb8nLy0uzZs3SCy+8oAULFqh9+/Y2yxs3bpycnZ31/vvv69ixY/rmm2+UM2dO2dnZKTw8XAEBAdq6dauCgoJUvHhxjRgxwmb+5OzzCUnOZ/N+/frp7NmzWr16tX766ad4y+jXr5+CgoLUq1cvvfXWWwoNDdWkSZO0e/fueJ+/0/Iz5+rVq9W1a1c1bdpUn3zyiSTp0KFD2rRpkwYPHizpfjYYN26c+vTpo9q1aysyMlI7d+7Url271KxZs5T/4bMyAxlSmzZtjNq1a1tfd+jQwejQoYNhb29vLF++3DAMw9i1a5chyVi8eLFx/fp1w8PDw+jbt6/Ncs6fP2+4u7vbtI8cOdJ48E9/9OhRw87Ozmjfvr0RExNjM39sbKxhGIZx8eJFw8HBwWjevLlNn0mTJhmSjBkzZljbGjVqZEgyfvnlF2tbSEiIIcmws7Mztm7dam1fuXKlIcmYOXOmtW3mzJmGJCM0NNTaVqxYMUOSsWHDBmvbxYsXDUdHR+Odd96xtr355puGxWIxdu/ebW27cuWKkSdPnnjLfJTPPvsswXn27NljSDL69Olj0z5s2DBDkvHXX3+luO7Ro0cbuXPnNo4cOWKzzPfff9+wt7c3Tp8+ney6kXm5u7sbAwcOTHR6mzZtjGLFisVrX7dunSHJ8PX1NW7dumUz7eHjPc7Dx9m1a9cMV1dXo06dOsbt27dt+sb9O5BUDQkdtw/Wtm7dOmtb3L8RU6ZMibecRo0aGY0aNYo3f7ly5Yzo6Ghr+1dffWVIMvbt22cYhmFER0cbXl5eRq1atYy7d+9a+wUFBRmSbJYJxOnatauRL18+4969e9a2c+fOGXZ2dsaoUaMMw4h/DH3xxReGJOPSpUuJLjclx8PDx6xhGMa4ceMMi8VinDp1ytqW2LGclISW/euvv8Z7X4pb9gsvvGDTd8CAAYYkY+/evdY2SYYkY+fOnda2U6dOGU5OTkb79u2tbb179zYKFChgXL582WaZXbp0Mdzd3a213bt3z+bYNgzDCA8PN5566injtddes2mXZIwcOTKZW4/MrHv37oadnZ2xY8eOeNNiY2ONIUOGGJKMjRs3WtuvX79uFC9e3PDx8bF+Vo075ipWrGjcuXPH2rdr166GxWIxWrVqZbPsunXrxnuPS+4+//Bxn5LP5gMHDkzw+N64caMhyZg9e7ZN+4oVK+K1p/VnzsGDBxtubm42/z4+rEqVKkabNm0SnY7/w/DyDKpBgwbatWuXbt68KUn6+++/1bp1a1WtWlUbN26UdP/st8VisQ7juHbtmrp27arLly9bf+zt7VWnTp0kh2n9/vvvio2N1YgRI2RnZ7tLxH0LvmbNGt25c0dDhgyx6dO3b1+5ubnFG87j4uKiLl26WF+XKVNGHh4eKleunM1w+Lj/P3HixCN/J+XLl1eDBg2sr729vVWmTBmbeVesWKG6devafDOeJ08evfLKK49cfnItW7ZMkvT222/btL/zzjuSFO93kZy658+frwYNGsjT09Pm7/fcc88pJiYm2UOYkLl5eHho27ZtOnv2bKrm79Gjh5ydnVM17+rVq3X9+nW9//778a5fNeP+Co6OjurVq1ey+/fq1cvmzH3cMRV3HO3cuVNXrlxR3759lSPH/w3ieuWVV+Tp6ZlGVSOr6dy5sy5evGgz3HvBggWKjY1V586dE5zHw8NDkrR48eI0ufznwWP25s2bunz5surVqyfDMLR79+40W3ZUVJQuX76sp59+WpK0a9eueP0HDhxo8/rNN9+U9H/ve3Hq1q2rGjVqWF8XLVpUL774olauXKmYmBgZhqHffvtNbdu2lWEYNu9rLVq0UEREhHX99vb21mM7NjZWV69e1b1791SzZs0Ea0TWFxsbq99//11t27ZVzZo14023WCxatmyZateurWeeecba7uLiotdff10nT57UwYMHbebp3r27zRnhOnXqyDAMvfbaazb96tSpo7CwMN27d8+m/VH7fEIe57N5nPnz58vd3V3NmjWzWUaNGjXk4uISbxlp+ZnTw8NDN2/eTHKouIeHhw4cOKCjR48+cluyO4aXZ1ANGjTQvXv3tGXLFhUpUkQXL15UgwYNdODAAZvQXb58eeXJk8e6sz/77LMJLs/NzS3RdR0/flx2dnYqX758on1OnTol6X54fpCDg4N8fX2t0+MULlw43gd1d3d3FSlSJF6bpHjXpSSkaNGi8do8PT1t5j116pTq1q0br1/JkiUfufzkOnXqlOzs7OItM3/+/PLw8Ij3u0hO3UePHtW///5rHb7+sIsXL6ZB5cjoPv30U/Xo0UNFihRRjRo11Lp1a3Xv3l2+vr7Jmr948eKpXvfx48clKclhcmmpUKFCKRr+/vBxFBek446juOPu4eMyR44cSV5ag+ytZcuWcnd319y5c9W0aVNJ94eWV61aVaVLl05wns6dO+uHH35Qnz599P7776tp06bq0KGDOnbsGO+L6+Q4ffq0RowYoSVLlsR7L4yIiEj5Rj3g6tWrCgwM1Jw5c+K9jyS07FKlStm8LlGihOzs7OJdm/5wP0kqXbq0bt26pUuXLsnOzk7Xrl3TtGnTNG3atARre7CeWbNm6fPPP1dISIju3r1rbX+cf9OQeV26dEmRkZFJvh+dOnXK5iROnHLlylmnPzj/w+8hcZ8/E/pcGhsbq4iICHl5eVnbH7XP58+fP970x/ls/uAyIiIilC9fvgSnP3xcp+VnzgEDBmjevHlq1aqVChUqpObNm8vPz08tW7a09h01apRefPFFlS5dWhUrVlTLli316quvqnLlyo/ctuyG0J1B1axZU05OTtqwYYOKFi2qfPnyqXTp0mrQoIEmT56s6Ohobdy40XrNSty37T/99FOCB/6DZ36eBHt7+xS1G4aR6mUmZ14zJPfsX3Lqjo2NVbNmzfTuu+8m2DexD3/IWvz8/NSgQQMtWrRIq1at0meffaZPPvlECxcuVKtWrR45f0JnuRPbTxP7Zj61UrqelJ6Rz2jHP7IGR0dHtWvXTosWLdLkyZN14cIFbdq0SR9//HGi8zg7O2vDhg1at26d/vzzT61YsUJz587Vs88+q1WrVsne3j7Zx0NMTIyaNWumq1ev6r333lPZsmWVO3du/ffff+rZs+djn0n38/PT5s2b9b///U9Vq1aVi4uLYmNj1bJly2QtO7WjXOKW3a1bN/Xo0SPBPnEfyn/++Wf17NlT7dq10//+9z/ly5dP9vb2GjdunPXLQOBxmfG59FHS4rN5bGys8uXLp9mzZyc4/eHgnJafOfPly6c9e/Zo5cqVWr58uZYvX66ZM2eqe/fumjVrliSpYcOGOn78uBYvXqxVq1bphx9+0BdffKEpU6aoT58+j9y+7ITQnUE5ODhY77xbtGhR61CRBg0aKDo6WrNnz9aFCxfUsGFDSfe/jZbuHyDPPfdcitZVokQJxcbG6uDBg4nesCTumdmHDx+2Oet2584dhYaGpnidZilWrFiCdzROqO1REvuwUaxYMcXGxuro0aPWb1Sl+zeNuXbtWqqeL16iRAnduHEjw/wekX4KFCigAQMGaMCAAbp48aKqV6+usWPHqlWrVqn6ABx3RvjatWvWYbGS4o3IiPs3ZP/+/UmODEmshgfX86CH12OWuOPu2LFjatKkibX93r17OnnyJN+6I1GdO3fWrFmztHbtWh06dEiGYSQ6tDyOnZ2dmjZtqqZNm2rixIn6+OOPNXz4cK1bt07PPfdcso+Hffv26ciRI5o1a5a6d+9ubU+LO/+Gh4dr7dq1CgwMtLkpVFLDQI8ePWpzdvnYsWOKjY2NN1okoWUcOXJEuXLlsoYAV1dXxcTEPPJ9bcGCBfL19dXChQtt/n0ZOXJkkvMh6/L29pabm5v279+faJ9ixYrp8OHD8dpDQkKs09NScvb5h6Xks3li760lSpTQmjVrVL9+/VRfPpbQMpP7mdPBwUFt27ZV27ZtFRsbqwEDBmjq1Kn66KOPrJ8V8uTJo169eqlXr166ceOGGjZsqICAAEL3Q7imOwNr0KCBtm3bpnXr1llDd968eVWuXDnrXQTj2lu0aCE3Nzd9/PHHNkOz4ly6dCnR9bRr1052dnYaNWpUvG++474Ze+655+Tg4KCvv/7a5tuy6dOnKyIiQm3atHm8jU0jLVq00JYtW7Rnzx5r29WrVxP9hjApuXPnlhT/Q1Pr1q0lKd4dxSdOnChJqfpd+Pn5acuWLVq5cmW8adeuXYt3bRGynpiYmHjDPfPly6eCBQtaH1+SO3fuFA83jXvTf/C+ADdv3rR+Sx2nefPmcnV11bhx4+I9IujBYz6xGhJaT0xMTKJDS9NazZo15eXlpe+//97meJk9e3ayLl9B9vXcc88pT548mjt3rubOnavatWsnOaz56tWr8drivrCOO1aTezzEnZV68BgzDMPmkTypldCypfjvXQ+Ke2xRnG+++UaS4o202bJli8311mFhYVq8eLGaN28ue3t72dvb66WXXtJvv/2WYHB68DNJQnVu27ZNW7ZsSWrzkIXZ2dmpXbt2+uOPP7Rz58540w3DUOvWrbV9+3ab/eTmzZuaNm2afHx8krxkMjUetc8nJCWfzRP7zOnn56eYmBiNHj063vz37t2L1z85kvuZ8+FHC9rZ2Vm/wI77t+7hPi4uLipZsmS8x66BM90ZWoMGDTR27FiFhYXZ3BShYcOGmjp1qnx8fKyPwXFzc9N3332nV199VdWrV1eXLl3k7e2t06dP688//1T9+vU1adKkBNdTsmRJDR8+XKNHj1aDBg3UoUMHOTo6aseOHSpYsKDGjRsnb29v+fv7KzAwUC1bttQLL7ygw4cPa/LkyapVq5a6dev2RH4nj/Luu+/q559/VrNmzfTmm29aHxlWtGhRXb16NUVnCuNumDF8+HB16dJFOXPmVNu2bVWlShX16NFD06ZN07Vr19SoUSNt375ds2bNUrt27WzOsiXX//73Py1ZskTPP/+89dEON2/e1L59+7RgwQKdPHnS+tgyZE3Xr19X4cKF1bFjR1WpUkUuLi5as2aNduzYoc8//1zS/X1y7ty5evvtt1WrVi25uLiobdu2SS63efPmKlq0qHr37q3//e9/sre314wZM6z/PsRxc3PTF198oT59+qhWrVp6+eWX5enpqb179+rWrVvWkJ5YDRUqVNDTTz8tf39/Xb16VXny5NGcOXOe2BdGDg4OCggI0Jtvvqlnn31Wfn5+OnnypIKCglSiRAlTbgaHrCFnzpzq0KGD5syZo5s3b2rChAlJ9h81apQ2bNigNm3aqFixYrp48aImT56swoULW2/qlNzjoWzZsipRooSGDRum//77T25ubvrtt9/S5IsiNzc3NWzYUJ9++qnu3r2rQoUKadWqVQoNDU10ntDQUL3wwgtq2bKltmzZop9//lkvv/yyqlSpYtOvYsWKatGihc3jk6T7jx+NM378eK1bt0516tRR3759Vb58eV29elW7du3SmjVrrF9ePP/881q4cKHat2+vNm3aKDQ0VFOmTFH58uV148aNx/49IHP6+OOPtWrVKjVq1Eivv/66ypUrp3Pnzmn+/Pn6+++/9f777+vXX39Vq1at9NZbbylPnjyaNWuWQkND9dtvv6Xq/gpJSc4+/7CUfDaP+8z51ltvqUWLFrK3t1eXLl3UqFEj9evXT+PGjdOePXvUvHlz5cyZU0ePHtX8+fP11VdfqWPHjinaluR+5uzTp4+uXr2qZ599VoULF9apU6f0zTffqGrVqtaRnuXLl1fjxo1Vo0YN5cmTRzt37rQ++hQPecJ3S0cKREZGGvb29oarq6vN7fp//vlnQ5Lx6quvxptn3bp1RosWLQx3d3fDycnJKFGihNGzZ0+bxxwk9tiRGTNmGNWqVTMcHR0NT09Po1GjRsbq1att+kyaNMkoW7askTNnTuOpp54y3njjDSM8PNymT6NGjYwKFSrEW36xYsUSfKyAJJvHJCX2yLCE5n348UKGYRi7d+82GjRoYDg6OhqFCxc2xo0bZ3z99deGJOP8+fPxlpGU0aNHG4UKFTLs7Oxsarp7964RGBhoFC9e3MiZM6dRpEgRw9/f34iKikrWNidU9/Xr1w1/f3+jZMmShoODg5E3b16jXr16xoQJE2wec4GsKTo62vjf//5nVKlSxXB1dTVy585tVKlSxZg8ebK1z40bN4yXX37Z8PDwMCRZH2sS90iU+fPnJ7jsf/75x6hTp47h4OBgFC1a1Jg4cWKijzRasmSJUa9ePcPZ2dlwc3Mzateubfz666+PrMEwDOP48ePGc889Zzg6OhpPPfWU8cEHHxirV69O8JFhCf0bETctoUeGPbxtoaGh8R43aBiG8fXXXxvFihUzHB0djdq1axubNm0yatSoYbRs2TLB9QGGYVj3U4vFYoSFhdlMe/g9c+3atcaLL75oFCxY0HBwcDAKFixodO3aNd7jd5J7PBw8eNB47rnnDBcXFyNv3rxG3759jb1798bbv1PzyLAzZ84Y7du3Nzw8PAx3d3ejU6dOxtmzZ+M9eitu2QcPHjQ6duxouLq6Gp6ensagQYPiPUIw7j37559/NkqVKmU4Ojoa1apVs9mmOBcuXDAGDhxoFClSxMiZM6eRP39+o2nTpsa0adOsfWJjY42PP/7YetxWq1bNWLp0qdGjR48EH93EI8Oyj1OnThndu3c3vL29DUdHR8PX19cYOHCg9RFzx48fNzp27Gh4eHgYTk5ORu3atY2lS5faLCOx95C498CHH0kWdyw8+EjA5O7zST0q8FGfze/du2e8+eabhre3t2GxWOId69OmTTNq1KhhODs7G66urkalSpWMd9991zh79qy1T1p/5lywYIHRvHlzI1++fNbPD/369TPOnTtnXc6YMWOM2rVrGx4eHoazs7NRtmxZY+zYsXxuTYDFMLgLDbK+IUOGaOrUqbpx40aiw4AAZD2xsbHy9vZWhw4d9P3336d3OUCGFBAQoMDAQF26dOmRo6osFosGDhyY6Og5IKthn0da4JpuZDm3b9+2eX3lyhX99NNPeuaZZwjcQBYWFRUV7/rVH3/8UVevXlXjxo3TpygAAJDtcU03spy6deuqcePGKleunC5cuKDp06crMjJSH330kSTpxo0bj7xOzNvbm4AOZDJbt27V0KFD1alTJ3l5eWnXrl2aPn26KlasqE6dOqV3eUCaiYiIiPcF88MSekQRACB9ELqR5bRu3VoLFizQtGnTZLFYVL16dU2fPt36eLUJEyYkeeML6f7NZB5+RAqAjM3Hx0dFihTR119/bb15Vffu3TV+/Hg5ODikd3lAmhk8eHC8JxA8jKsHASDj4JpuZDsnTpzQiRMnkuzzzDPPyMnJ6QlVBABA8h08eFBnz55Nsk9ynsELAHgyCN0AAAAAAJiEG6kBAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACb5f0GDQGF7BlLPAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import json\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "\n",
    "# 设置中文字体\n",
    "plt.rcParams['font.sans-serif'] = ['SimHei']\n",
    "plt.rcParams['axes.unicode_minus'] = False\n",
    "\n",
    "# 三个质量差异明显的典型回答样本\n",
    "# 差：只是简单罗列信息，没有结构和感情\n",
    "poor_response = \"公司福利：健康保险，家属可用。15天年假，5天病假。交通补贴500，餐饮补贴300。培训基金8000。健身房有折扣。\"\n",
    "\n",
    "# 中：有基本结构和分类，但语气平淡\n",
    "medium_response = \"\"\"\n",
    "公司福利：\n",
    "1. 💦健康和假期：\n",
    "   - 健康保险（含家属）\n",
    "   - 年度体检\n",
    "   - 15天年假和5天病假\n",
    "2. 💰补贴和激励：\n",
    "   - 每月500交通补贴和300餐饮补贴\n",
    "   - 8000元/年的教育培训基金\n",
    "   - 合作健身房折扣\n",
    "\"\"\"\n",
    "\n",
    "# 优：结构清晰、语气友好、视觉吸引力强（直接使用我们的参考答案）\n",
    "good_response = reference_answer \n",
    "\n",
    "# 设计更细化的评估维度\n",
    "def grade_response_detailed(response_to_grade):\n",
    "    grader_prompt = f\"\"\"\n",
    "    【角色】你是一位经验丰富的内部沟通和员工体验评测官。\n",
    "    【任务】请根据以下四个维度，对提供的“公司福利介绍”文本进行1-5分的量化评分。\n",
    "    \n",
    "    【评分维度】\n",
    "    1.  **欢迎语气 (welcoming_tone)**: 1分表示语气冰冷生硬，5分表示非常热情、有感染力。\n",
    "    2.  **内容结构化 (structuring)**: 1分表示信息混乱无序，5分表示分类清晰、逻辑性强。\n",
    "    3.  **视觉吸引力 (visual_appeal)**: 1分表示枯燥乏味，5分表示善用表情符号、粗体等元素，非常吸引眼球。\n",
    "    4.  **信息完整性 (completeness)**: 1分表示信息缺失严重，5分表示关键福利信息完整无缺。\n",
    "\n",
    "    【待评估文本】\n",
    "    {response_to_grade}\n",
    "    ---\n",
    "    【输出要求】\n",
    "    请严格以JSON格式返回你的评分，不要包含任何解释。例如：\n",
    "    {{\"welcoming_tone\": 5, \"structuring\": 4, \"visual_appeal\": 5, \"completeness\": 5}}\n",
    "    \"\"\"\n",
    "    try:\n",
    "        raw_output = llm.invoke(grader_prompt)\n",
    "        # 提取JSON部分\n",
    "        json_str = raw_output[raw_output.find('{'):raw_output.rfind('}')+1]\n",
    "        return json.loads(json_str)\n",
    "    except (json.JSONDecodeError, IndexError):\n",
    "        # 容错处理，在无法解析时返回一个默认的低分\n",
    "        return {{\"welcoming_tone\": 1, \"structuring\": 1, \"visual_appeal\": 1, \"completeness\": 1}}\n",
    "\n",
    "# 对三个典型样本进行评分\n",
    "# 注意：这里的 key 将作为图表中的标签，我们使用新视觉方案中提供的名称\n",
    "scores = {\n",
    "    \"Original Answer\": grade_response_detailed(poor_response),\n",
    "    \"Single Iteration Optimize\": grade_response_detailed(medium_response),\n",
    "    \"Multi-turn Iteration Optimize\": grade_response_detailed(good_response)\n",
    "}\n",
    "\n",
    "# 将 scores 转换为 DataFrame\n",
    "df = pd.DataFrame(scores)\n",
    "df = df.reset_index().rename(columns={'index': 'Dim'})\n",
    "df_long = df.melt(id_vars='Dim', var_name='Version', value_name='Score')\n",
    "\n",
    "\n",
    "# --- 单幅分组柱状图 ---\n",
    "plt.figure(figsize=(10, 6))\n",
    "ax = sns.barplot(\n",
    "    data=df_long,\n",
    "    x=\"Dim\",        # 每个维度一组\n",
    "    y=\"Score\",\n",
    "    hue=\"Version\",  # 3 个版本并排\n",
    "    palette=\"viridis\"\n",
    ")\n",
    "\n",
    "# 为每根柱子添加数值标签\n",
    "for p in ax.patches:\n",
    "    height = p.get_height()\n",
    "    if height == 0:          # 跳过高度为 0 的占位 patch\n",
    "        continue\n",
    "    ax.annotate(\n",
    "        f\"{height}\",\n",
    "        (p.get_x() + p.get_width() / 2., height),\n",
    "        ha='center', va='center',\n",
    "        xytext=(0, 5),\n",
    "        textcoords='offset points',\n",
    "        fontsize=11\n",
    "    )\n",
    "\n",
    "# 轴和标题美化\n",
    "ax.set_ylim(0, 6)\n",
    "ax.set_ylabel('Score (1-5)', fontsize=12)\n",
    "ax.set_xlabel('')                # 隐藏 X 轴标题\n",
    "ax.set_title('Evaluation', fontsize=20)\n",
    "ax.tick_params(axis='x', labelsize=12)\n",
    "\n",
    "plt.legend()       # 显示图例\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61t4506kjip",
   "metadata": {},
   "source": [
    "你可以看到，每次迭代的大模型的回复效果都有不同程度的提升。通过这种量化评估，你不仅能直观地看到每次优化带来的提升，还能将“好”或“不好”这种模糊的感觉，转化为清晰、可衡量的数据。这为你提供了一种科学的方法来验证和迭代你的提示词策略，确保每一步改进都有据可依，最终交付出真正高质量的用户体验。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7c107a1",
   "metadata": {},
   "source": [
    "## 5. 推理大模型\n",
    "\n",
    "前面所讲的提示词技巧和提示词框架可以广泛适用于通用大模型（如Qwen2.5-max、GPT-4、DeepSeek-V3），这类模型面向通用对话、知识问答、文本生成等广泛的场景。除了通用大模型，目前还有一类专门为“推理”设计的大模型——`推理大模型`。\n",
    "\n",
    "### 5.1 什么是推理大模型?\n",
    "\n",
    "你可能已经通过2.1节的扩展阅读了解了阿里云的推理大模型[Qwen3](https://help.aliyun.com/zh/model-studio/deep-thinking) 。为了方便本节后续调用，改写了2.1节中提供的代码，你可以尝试运行以下代码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "500b6e30",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "import os\n",
    "\n",
    "def reasoning_model_response(user_prompt, system_prompt=\"你是一个编程助手。\", model=\"qwen3-235b-a22b-thinking-2507\"):\n",
    "    \"\"\"\n",
    "    prompt: 用户输入的提示词\n",
    "    model: 此处以 qwen3-235b-a22b-thinking-2507 为例，可按需更换推理模型名称，如：deepseek-r1\n",
    "    \"\"\"\n",
    "    # 初始化客户端\n",
    "    client = OpenAI(\n",
    "        api_key=os.getenv(\"DASHSCOPE_API_KEY\"),\n",
    "        base_url=os.getenv(\"BASE_URL\")\n",
    "    )\n",
    "\n",
    "    # 初始化状态变量\n",
    "    is_answering = False\n",
    "\n",
    "    # 发起流式请求\n",
    "    completion = client.chat.completions.create(\n",
    "        model=model,\n",
    "        # messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": user_prompt}\n",
    "        ],\n",
    "        stream=True,\n",
    "    )\n",
    "\n",
    "    # 打印思考过程标题\n",
    "    print(\"\\n\" + \"=\" * 20 + \"思考过程\" + \"=\" * 20 + \"\\n\")\n",
    "\n",
    "    # 处理流式响应\n",
    "    for chunk in completion:\n",
    "        if chunk.choices:\n",
    "            delta = chunk.choices[0].delta\n",
    "            if hasattr(delta, 'reasoning_content') and delta.reasoning_content is not None:\n",
    "                # 处理思考过程内容\n",
    "                print(delta.reasoning_content, end='', flush=True)\n",
    "            else:\n",
    "                # 切换到答案输出模式\n",
    "                if delta.content != \"\" and not is_answering:\n",
    "                    print(\"\\n\" + \"=\" * 20 + \"完整回复\" + \"=\" * 20 + \"\\n\")\n",
    "                    is_answering = True\n",
    "                # 处理答案内容\n",
    "                if delta.content:\n",
    "                    print(delta.content, end='', flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e65df7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "reasoning_model_response(user_prompt=\"你是谁？\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fea9f93",
   "metadata": {},
   "source": [
    "\n",
    "通过例子可以看到，推理模型相较于通用大模型多出了“`思考过程`”，就像解数学题时有人会先在草稿纸上一步步推导，而不是直接报答案，减少模型出现“拍脑袋”的错误，同时在分步思考过程中，如果某一步骤发现矛盾，还可以回头检查并重新调整思路，展示推理步骤还可以方便人们理解，顺着模型的思考路线验证逻辑。<br>\n",
    "相较于通用大模型，推理大模型通常在解决复杂问题时更可靠，比如在数学解题、代码编写、法律案件分析等需要严谨推理的场景。并不是说推理模型一定更好，两种模型都有各自的应用场景，下表从一些典型维度对这两类模型进行了对比：<br>\n",
    "\n",
    "| 维度            | 推理模型  | 通用模型  |\n",
    "|-------------------|------------------|------------|\n",
    "| 设计目标     | 专注于**逻辑推理、多步问题求解、数学计算**等需要深度分析的任务 | 面向**通用对话、知识问答、文本生成**等广泛场景 |\n",
    "| 训练数据侧重  | 大量**数学题解、代码逻辑、科学推理**数据集增强推理能力 | 覆盖**百科、文学、对话**等多领域海量数据 |\n",
    "| 典型输出特征 | 输出包含**完整推导**步骤，注重逻辑链条的完整性 | 输出**简洁直接**，侧重结果的自然语言表达 |\n",
    "| 响应速度 | 复杂推理任务**响应较慢**（需多步计算） | 常规任务**响应更快**（单步生成为主） |\n",
    "\n",
    "推理模型还是通用模型？如何选择？以下是一些推荐：\n",
    "- **明确的通用任务**：对于明确定义的问题，**通用模型**一般能够很好地处理。\n",
    "- **复杂任务**：对于非常复杂的任务，且需要给出相对**更精确和可靠**的答案，推荐使用**推理模型**。这些任务可能有：\n",
    "    - 模糊的任务：任务相关信息很少，你无法提供模型相对明确的指引。\n",
    "    - 大海捞针：传递大量非结构化数据，提取最相关的信息或寻找关联/差别。\n",
    "    - 调试和改进代码：需要审查并进一步调试、改进大量代码。\n",
    "- **速度和成本**：一般来说推理模型的推理时间较长，如果你对于时间和成本敏感，且任务复杂度不高，**通用模型**可能是更好的选择。<br>\n",
    "\n",
    "当然你还可以在你的应用中结合使用两种模型：使用推理模型完成Agent的规划和决策，使用通用模型完成任务执行。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "580c0b84",
   "metadata": {},
   "source": [
    "### 5.2 适用于推理大模型的提示词技巧\n",
    "\n",
    "推理模型在面对相对模糊的任务也能给出详尽且格式良好的响应。你依然可以通过**提示词技巧**保证推理大模型的推理质量的下限：<br>\n",
    "#### 技巧一：保持任务提示简洁清晰，提供足够的背景信息<br>\n",
    "本节4.1中介绍的**清晰表达需求**同样适用于推理模型，虽然推理模型能力很强，但却不能“看穿人的想法”，你需要保持提示**简洁、清晰**，从而让推理大模型专注于核心任务。<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe5cc8ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "bad_prompt=\"\"\"\n",
    "def example(a):\n",
    "  b = []\n",
    "  for i in range(len(a)):\n",
    "    b.append(a[i]*2)\n",
    "  return sum(b)\n",
    "\"\"\"\n",
    "\n",
    "reasoning_model_response(user_prompt=bad_prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61e6f145",
   "metadata": {},
   "source": [
    "通过如上示例，可以看到就算你只给推理大模型一段代码，它还是能够通过一系列的推理给出很丰富的答案，但返回的推理中可能包含了很多你不关注的信息，你可以尝试明确**任务目标**，从而获得更有针对性的建议："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10c90bd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_A=\"\"\"\n",
    "如下 python 代码有什么问题？怎么优化？\n",
    "def example(a):\n",
    "  b = []\n",
    "  for i in range(len(a)):\n",
    "    b.append(a[i]*2)\n",
    "  return sum(b)\n",
    "\"\"\"\n",
    "\n",
    "reasoning_model_response(user_prompt=prompt_A)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5500ab0",
   "metadata": {},
   "source": [
    "同样地，你还可以在结合本节**4.2 限定角色和受众**、**4.3规定输出格式**等技巧进一步限定范围，确保结果符合你的预期。<br>\n",
    "同时，如果提示词比较复杂，你可以通过**分隔符**帮助模型理解你的意图。<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0d81507",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_B=\"\"\"\n",
    "<audience>初级Python开发者</audience>\n",
    "\n",
    "<task>函数性能优化，优化code中的代码。</task>\n",
    "\n",
    "<format>\n",
    "如有多种优化方案请按照如下格式进行输出：\n",
    "【优化方案X】\n",
    "问题描述：[描述]\n",
    "优化方案：[描述]\n",
    "示例代码：[代码块]\n",
    "</format>\n",
    "\n",
    "<code>\n",
    "def example(a):\n",
    "  b = []\n",
    "  for i in range(len(a)):\n",
    "    b.append(a[i]*2)\n",
    "  return sum(b)\n",
    "</code>\n",
    "\"\"\"\n",
    "\n",
    "reasoning_model_response(user_prompt=prompt_B)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l78dh3zta8",
   "metadata": {},
   "source": [
    "#### 技巧二：避免思维链提示\n",
    "本节 4.5 中你了解到通过思维链（COT）技术让大模型深入思考提升回复效果。\n",
    "一般来说，你无需提示推理模型“逐步思考”或“解释你的推理”，因为它们本身会进行深入的思考，你的提示可能反而限制推理模型的发挥。除非你需要大模型严格按照固定的思路去推理，这种情况很少发生。\n",
    "\n",
    "#### 技巧三：根据模型响应调整提示词\n",
    "推理模型因其回复形式(包含**思考过程**)，天然适合你分析它的思考推理结论的过程，便于你调整提示词。\n",
    "因此，你不需要纠结提示词是否足够完善，只需要不断与推理模型对话，过程中补充信息，完善提示词即可。\n",
    "比如当你的描述**太抽象**或**无法准确描述**时，你可以用本节4.4讲到的**增加示例**的技巧来明确这些信息，这些示例有时可以从与模型的对话历史中挑选出来。\n",
    "这个过程可以是重复多次的，不断尝试调整提示，让模型不断推理迭代，直到符合你的要求。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c163ae4c",
   "metadata": {},
   "source": [
    "#### 技巧四：让推理模型成为你的“提示词教练”\n",
    "\n",
    "在 4.6 节中，你学习了 **Meta Prompting**，即让大模型帮助你优化提示词。那么，哪种模型最适合扮演这位“教练”的角色呢？\n",
    "\n",
    "答案是**推理模型**。\n",
    "\n",
    "由于推理模型擅长分步思考和逻辑推导，它们在分析一个提示词的优缺点、并系统性地提出改进建议方面表现得尤为出色。它们不仅能给你一个更好的提示词，还能清晰地展示出“为什么”这样改会更好，让你在优化过程中也能学到提示词工程的精髓。\n",
    "\n",
    "让我们重新审视 4.6 节中的例子，但这次，我们使用在 6.1 节中定义的 `reasoning_model_response` 函数来充当我们的提示词教练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01807cca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 我们将复用 4.6 节中的公司福利场景\n",
    "retrieved_text = \"\"\"\n",
    "关于公司的福利政策，我们提供全面的健康保险，覆盖员工及其直系家属。\n",
    "年度体检是标配。此外，每年有15天的带薪年假，以及5天的带薪病假。\n",
    "我们还提供每月500元的交通补贴和300元的餐饮补贴。\n",
    "为了鼓励员工成长，公司设有每年高达8000元的教育培训基金，员工可以申请用于课程学习或购买专业书籍。\n",
    "健身方面，公司与多家健身房有合作，员工可享受折扣价。\n",
    "\"\"\"\n",
    "\n",
    "# 初始的、比较简单的提示词\n",
    "initial_prompt = f\"\"\"\n",
    "根据以下信息，回答新员工关于公司福利的问题。\n",
    "\n",
    "【参考信息】\n",
    "{retrieved_text}\n",
    "\"\"\"\n",
    "\n",
    "# 假设这是通用模型的、不甚理想的输出\n",
    "initial_response = \"\"\"\n",
    "我们公司提供全面的健康保险，覆盖员工及其家属。每年有15天带薪年假和5天病假。还有每月500元交通补贴和300元餐饮补贴。公司提供8000元的年度教育基金，并与健身房有合作折扣。\n",
    "\"\"\"\n",
    "\n",
    "# 构建我们的 Meta Prompt，请求推理模型帮助优化\n",
    "meta_prompt_for_reasoning = f\"\"\"\n",
    "我正在为公司的新员工答疑机器人优化一个提示词，目标是回答关于“公司福利”的问题。\n",
    "\n",
    "这是我的第一个尝试：\n",
    "---\\n{initial_prompt}\\n---\n",
    "\n",
    "这是它生成的输出：\n",
    "---\\n{initial_response}\\n---\n",
    "\n",
    "这个输出不够好。我希望机器人的回答更具吸引力，并且结构清晰，能让新员工快速抓住重点。具体要求如下：\n",
    "1.  **语气**：友好、热情，有欢迎新同事的感觉。\n",
    "2.  **结构**：使用清晰的要点（比如用表情符号开头的列表）来组织内容。\n",
    "3.  **内容**：将福利分为几个类别，如“健康与假期”、“补贴与激励”等。\n",
    "\n",
    "请你扮演一位提示词工程专家，帮我重写这个提示词，以实现上述目标。\n",
    "请在最终答案中，只给出优化后的提示词本身，不要包含其他解释性文字。\n",
    "\"\"\"\n",
    "\n",
    "# 使用推理模型来获取优化建议\n",
    "reasoning_model_response(user_prompt=meta_prompt_for_reasoning, system_prompt=\"你是一位顶级的提示词工程专家。\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cf7690c",
   "metadata": {},
   "source": [
    "通过上面的示例，你可以清晰地看到推理模型是如何一步步分析原始提示词的不足，并结合我们提出的要求（友好的语气、清晰的结构、生动的内容等），最终构建出一个结构更加严谨、意图更加明确的优化版提示词。\n",
    "\n",
    "这充分展示了将合适的工具用在合适的任务上的重要性，这背后是 **任务复杂性** 与 **执行成本（时间与费用）** 之间的权衡：\n",
    "\n",
    "- **通用模型（非推理模型）**：\n",
    "    - **优势**：执行速度快，成本较低。\n",
    "    - **适用场景**：适合执行相对直接、明确的任务，例如根据一个已经优化好的提示词进行信息提取、格式转换或简单问答。\n",
    "\n",
    "- **推理模型**：\n",
    "    - **优势**：擅长处理复杂、模糊或需要深度逻辑推导的任务。\n",
    "    - **适用场景**：更适合执行“元任务”（meta-task），例如分析和优化另一个任务（即提示词本身）的定义、进行复杂的规划或调试代码。\n",
    "    - **成本**：由于需要进行多步思考，其响应时间通常更长，成本也相对更高。\n",
    "\n",
    "在你构建自己的大模型应用时，**混合使用这两种模型**往往是实现最佳性价比的策略。你可以借鉴这种思路，构建一个“分工明确”的智能系统：\n",
    "当任务需要深度思考或规划时，引入推理模型来充当“**规划师**”或“**分析师**”的角色，让它来分解复杂任务或优化流程；然后，将拆解后的、更简单的子任务交由**通用模型**或其他工具来高效、低成本地执行。这种协作模式，能让你在保证高质量输出的同时，有效控制应用的响应时间和运行成本。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30451423",
   "metadata": {},
   "source": [
    "## ✅ 本节小结\n",
    "\n",
    "通过学习本节课程，你掌握了更多**上下文工程（Context Engineering）** 的高级技巧，从设计精细的提示词到实现智能的意图路由。回顾过去两节课，我们已经为答疑机器人装备了两大核心能力：<br>\n",
    "1. **知识填充 (通过 RAG)**：解决了模型“不知道”私有信息的问题。\n",
    "2. **行为引导 (通过提示词与控制流)**：解决了模型“如何做”、“做什么”的问题。\n",
    "这些技巧的共同核心，正是上下文工程的精髓：<br>\n",
    "通过精心设计和填充模型的上下文窗口，来引导和控制其生成期望的输出——这对于你用好大模型至关重要。\n",
    "\n",
    "实际落地大模型应用的过程中，提示词部分经常让领域专家来共同设计。因此，在你的工程代码中硬编码提示词，应该考虑调整成可配置的，甚至应用流程也可配置，这样能更方便领域专家参与提示词和整个流程的设计。阿里云百炼提供了可视化的大模型[应用构建](https://help.aliyun.com/zh/model-studio/user-guide/application-introduction#7c79befb2djg9)能力，可以在页面上完成提示词编写、整个复杂应用的流程可视化搭建，非常适合在需要非技术背景的领域专家参与的大模型应用开发项目中。\n",
    "\n",
    "在下一节中，你将学习自动化评测方法测试答疑机器人的表现，你可以使用量化的指标来评测你通过上下文工程所做的优化效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3166b70c",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## 🔥 课后小测验\n",
    "\n",
    "### 🔍 单选题\n",
    "<details>\n",
    "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
    "<b>以下哪个提示词要素用于明确要求大模型完成的任务❓</b>\n",
    "\n",
    "- A. 角色 (Role)     \n",
    "- B. 受众 (Audience)   \n",
    "- C. 任务目标 (Object)  \n",
    "- D. 上下文 (Context)  \n",
    "\n",
    "**【点击查看答案】**\n",
    "</summary>\n",
    "\n",
    "<div style=\"margin-top: 10px; padding: 15px;  border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
    "\n",
    "✅ **参考答案：C**  \n",
    "📝 **解析**：\n",
    "- 任务目标 (Object) 明确规定了大模型需要执行的操作或达成的结果。其他选项并非直接定义任务本身。\n",
    "- 角色 (Role) 定义大模型扮演的身份，受众 (Audience) 定义目标群体，上下文 (Context) 提供背景信息。\n",
    "\n",
    "\n",
    "</div>\n",
    "</details>\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "### 🔍 多选题\n",
    "<details>\n",
    "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
    "<b>你接手了一个复杂的、用于生成代码的提示词，但缺乏相关文档。在直接修改这个“黑盒”提示词之前，你会执行哪些准备工作来确保后续优化的有效性？</b>\n",
    "\n",
    "- A. 收集模型输出不佳的具体案例（Bad Cases）。\n",
    "- B. 立即开始草拟一个全新的、更简单的替代版本。\n",
    "- C. 清晰地用文字描述出期望的输出标准和格式。\n",
    "- D. 通读整个提示词，尝试凭直觉找出逻辑漏洞。\n",
    "\n",
    "**【点击查看答案】**\n",
    "</summary>\n",
    "\n",
    "<div style=\"margin-top: 10px; padding: 15px; border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
    "\n",
    "✅ **参考答案：A,C**  \n",
    "📝 **解析**：\n",
    "本题考查在启动一个结构化优化流程前，必须具备哪些核心输入。正确的优化（如Meta Prompting）需要明确的“问题”和“目标”。\n",
    "- 选项A（坏样本）和选项C（期望输出）是定义“问题”和“目标”的关键步骤，是Meta Prompting方法的两个核心输入。\n",
    "- 选项B（推倒重来）和选项D（凭直觉修改）都是高风险低效率的方法，它们没有考虑系统的结构，也没有考虑系统的效果。\"\n",
    "\n",
    "</div>\n",
    "</details>\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "### 🔍 多选题\n",
    "<details>\n",
    "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
    "<b>在使用推理型大模型（如qwen3-235b-a22b-thinking-2507）执行深度思考任务时。以下哪些提示词技巧是推荐的❓</b>\n",
    "\n",
    "- A. 给出简单、明确的任务指示    \n",
    "- B. 补充足够的背景信息     \n",
    "- C. 如有特定用户群体或特定任务要求，设定角色或受众     \n",
    "- D. 必须在提示词里写“一步一步推理”或者“给出解释”，模型才会使用深度思考模式\n",
    "- E. 当描述太抽象或无法准确描述时，可以通过增加示例来明确这些信息 \n",
    "\n",
    "**【点击查看答案】**\n",
    "</summary>\n",
    "\n",
    "<div style=\"margin-top: 10px; padding: 15px; border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
    "\n",
    "✅ **参考答案：ABCE**  \n",
    "📝 **解析**：\n",
    "- 通常情况下，推理型大模型（如qwen3-235b-a22b-thinking-2507）无需显式要求即可通过类似“思维链”（chain of thought）的模式做深度思考。\n",
    "\n",
    "</div>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b354883f",
   "metadata": {},
   "source": [
    "## ✉️ 评价反馈\n",
    "\n",
    "感谢你学习阿里云大模型ACP认证课程，如果你觉得课程有哪里写得好、哪里写得不好，期待你[通过问卷提交评价和反馈](https://survey.aliyun.com/apps/zhiliao/Mo5O9vuie)。\n",
    "\n",
    "你的批评和鼓励都是我们前进的动力。"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
